[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:23:32 dist-test-slave-5rhg krb5kdc[26117](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:23:32 dist-test-slave-5rhg krb5kdc[26117](info): set up 2 sockets
Jun 19 08:23:32 dist-test-slave-5rhg krb5kdc[26117](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 08:23:33.556767 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:23:33 dist-test-slave-5rhg krb5kdc[26117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:23:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:23:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:23:33.673062 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:44727
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:42515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:44727
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:33.756425 26130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:33.756645 26130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:33.756669 26130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:33.757776 26130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:23:33.757812 26130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:33.757827 26130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:33.757839 26130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:23:33.757851 26130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:23:33.759259 26130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:44727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:44727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:33.759476 26130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:33.759647 26130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:33.764963 26135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:33.764964 26136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:33.764964 26138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:33.764997 26130 server_base.cc:1034] running on GCE node
I20260619 08:23:33.765475 26130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:33.765772 26130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:33.766978 26130 hybrid_clock.cc:648] HybridClock initialized: now 1781857413766959 us; error 43 us; skew 500 ppm
Jun 19 08:23:33 dist-test-slave-5rhg krb5kdc[26117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857413, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:33.769769 26130 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:23:33.770221 26130 webserver.cc:460] Webserver started at http://127.25.127.190:44277/ using document root <none> and password file <none>
I20260619 08:23:33.770465 26130 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:33.770532 26130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:33.770684 26130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:33.771797 26130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "ffb91fe58adc49fbba2cf6cd6d6d5e8c"
format_stamp: "Formatted at 2026-06-19 08:23:33 on dist-test-slave-5rhg"
server_key: "a488aa900812fd1a5015ac5cfc8de067"
server_key_iv: "7dff1534bd63248aadff4a81b9be9ee6"
server_key_version: "encryptionkey@0"
I20260619 08:23:33.772274 26130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "ffb91fe58adc49fbba2cf6cd6d6d5e8c"
format_stamp: "Formatted at 2026-06-19 08:23:33 on dist-test-slave-5rhg"
server_key: "a488aa900812fd1a5015ac5cfc8de067"
server_key_iv: "7dff1534bd63248aadff4a81b9be9ee6"
server_key_version: "encryptionkey@0"
I20260619 08:23:33.773614 26130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:33.774468 26145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:33.774664 26130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:33.774763 26130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "ffb91fe58adc49fbba2cf6cd6d6d5e8c"
format_stamp: "Formatted at 2026-06-19 08:23:33 on dist-test-slave-5rhg"
server_key: "a488aa900812fd1a5015ac5cfc8de067"
server_key_iv: "7dff1534bd63248aadff4a81b9be9ee6"
server_key_version: "encryptionkey@0"
I20260619 08:23:33.774893 26130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:33.786939 26130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:33.789543 26130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:33.789724 26130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:33.794525 26130 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:44727
I20260619 08:23:33.794562 26197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:44727 every 8 connection(s)
I20260619 08:23:33.794932 26130 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:23:33.795847 26198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:33.797634 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 26130
I20260619 08:23:33.797748 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:23:33.798336 26198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c: Bootstrap starting.
I20260619 08:23:33.798405 26110 external_mini_cluster.cc:1428] Setting key 8ea280ba2238d7307a3f8676d6a7ca4d
I20260619 08:23:33.799095 26198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:33.799523 26198 log.cc:824] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:33.800333 26198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c: No bootstrap required, opened a new log
I20260619 08:23:33.802127 26198 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb91fe58adc49fbba2cf6cd6d6d5e8c" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44727 } }
I20260619 08:23:33.802289 26198 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:33.802351 26198 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffb91fe58adc49fbba2cf6cd6d6d5e8c, State: Initialized, Role: FOLLOWER
I20260619 08:23:33.802512 26198 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb91fe58adc49fbba2cf6cd6d6d5e8c" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44727 } }
I20260619 08:23:33.802609 26198 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:23:33.802654 26198 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:23:33.802702 26198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:33.803520 26198 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb91fe58adc49fbba2cf6cd6d6d5e8c" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44727 } }
I20260619 08:23:33.803642 26198 leader_election.cc:302] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffb91fe58adc49fbba2cf6cd6d6d5e8c; no voters:
I20260619 08:23:33.803845 26198 leader_election.cc:288] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:23:33.803870 26203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:33.803987 26203 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c [term 1 LEADER]: Becoming Leader. State: Replica: ffb91fe58adc49fbba2cf6cd6d6d5e8c, State: Running, Role: LEADER
I20260619 08:23:33.804092 26203 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb91fe58adc49fbba2cf6cd6d6d5e8c" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44727 } }
I20260619 08:23:33.804124 26198 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:33.804458 26204 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffb91fe58adc49fbba2cf6cd6d6d5e8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb91fe58adc49fbba2cf6cd6d6d5e8c" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44727 } } }
I20260619 08:23:33.804570 26204 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:33.804721 26205 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffb91fe58adc49fbba2cf6cd6d6d5e8c. Latest consensus state: current_term: 1 leader_uuid: "ffb91fe58adc49fbba2cf6cd6d6d5e8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb91fe58adc49fbba2cf6cd6d6d5e8c" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44727 } } }
I20260619 08:23:33.804771 26205 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:33.805553 26208 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:23:33.806146 26208 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:23:33.808193 26208 catalog_manager.cc:1347] Generated new cluster ID: 42f2d397ed674a85af6becbb7d62b072
I20260619 08:23:33.808244 26208 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:23:33 dist-test-slave-5rhg krb5kdc[26117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:33.816658 26200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:33.799292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45850 (local address 127.25.127.190:44727)
0619 08:23:33.799529 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:33.799539 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:33.799784 (+ 245us) server_negotiation.cc:407] Connection header received
0619 08:23:33.802114 (+ 2330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:33.802120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:33.802166 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:33.802305 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:33.804047 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:33.805658 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:33.807090 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:33.807593 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:33.807709 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:33.812430 (+ 4721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:33.812433 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:33.812434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:33.812445 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:33.814551 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:33.815077 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:33.815079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:33.815080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:33.815204 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:33.815534 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:33.815535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:33.815535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:33.815818 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:33.815958 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:33.816291 (+ 333us) server_negotiation.cc:299] Negotiation successful
0619 08:23:33.816422 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260619 08:23:33.823451 26208 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:23:33.824237 26208 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:23:33.835268 26208 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffb91fe58adc49fbba2cf6cd6d6d5e8c: Generated new TSK 0
I20260619 08:23:33.835505 26208 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
I20260619 08:23:33.860651 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:44727
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:42515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:33.948540 26224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:33.948746 26224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:33.948771 26224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:33.949870 26224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:33.949906 26224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:33.949932 26224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:23:33.952044 26224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:44727
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:33.952344 26224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:33.952620 26224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:33.957821 26232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:33.957823 26229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:33.957880 26230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:33.958109 26224 server_base.cc:1034] running on GCE node
I20260619 08:23:33.958313 26224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:33.958563 26224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:33.959739 26224 hybrid_clock.cc:648] HybridClock initialized: now 1781857413959723 us; error 29 us; skew 500 ppm
Jun 19 08:23:33 dist-test-slave-5rhg krb5kdc[26117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857413, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:33.962327 26224 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:23:33.962831 26224 webserver.cc:460] Webserver started at http://127.25.127.129:43335/ using document root <none> and password file <none>
I20260619 08:23:33.963076 26224 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:33.963141 26224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:33.963276 26224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:33.964462 26224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "1b1747bb9e4d4a5fb3e89d816ec4136b"
format_stamp: "Formatted at 2026-06-19 08:23:33 on dist-test-slave-5rhg"
server_key: "b358f6f21cfa73fa7b48e5b4172df322"
server_key_iv: "948723ddfe637920477765fe72f9dea4"
server_key_version: "encryptionkey@0"
I20260619 08:23:33.964828 26224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "1b1747bb9e4d4a5fb3e89d816ec4136b"
format_stamp: "Formatted at 2026-06-19 08:23:33 on dist-test-slave-5rhg"
server_key: "b358f6f21cfa73fa7b48e5b4172df322"
server_key_iv: "948723ddfe637920477765fe72f9dea4"
server_key_version: "encryptionkey@0"
I20260619 08:23:33.966159 26224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:33.966908 26239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:33.967079 26224 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:33.967173 26224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "1b1747bb9e4d4a5fb3e89d816ec4136b"
format_stamp: "Formatted at 2026-06-19 08:23:33 on dist-test-slave-5rhg"
server_key: "b358f6f21cfa73fa7b48e5b4172df322"
server_key_iv: "948723ddfe637920477765fe72f9dea4"
server_key_version: "encryptionkey@0"
I20260619 08:23:33.967253 26224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:33.978551 26224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:33.980533 26224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:33.980687 26224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:33.981014 26224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:33.981328 26224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:33.981380 26224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:33.981420 26224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:33.981444 26224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:33.989214 26224 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:42665
I20260619 08:23:33.989282 26352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:42665 every 8 connection(s)
I20260619 08:23:33.989648 26224 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:23:33.995584 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 26224
I20260619 08:23:33.995694 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
Jun 19 08:23:33 dist-test-slave-5rhg krb5kdc[26117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857413, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:33.995857 26110 external_mini_cluster.cc:1428] Setting key 9972dcd836d059d05162cf9e3d07d908
I20260619 08:23:33.999415 26200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:33.990388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:43993 (local address 127.25.127.190:44727)
0619 08:23:33.990472 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:33.990474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:33.990640 (+ 166us) server_negotiation.cc:407] Connection header received
0619 08:23:33.991181 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:33.991182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:33.991198 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:33.991251 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:33.992314 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:33.992812 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:33.993843 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:33.994046 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:33.994096 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:33.996257 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:33.996258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:33.996258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:33.996264 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:33.997936 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:33.998460 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:33.998461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:33.998461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:33.998523 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:33.998801 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:33.998801 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:33.998802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:33.998906 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:33.998954 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:33.999281 (+ 327us) server_negotiation.cc:299] Negotiation successful
0619 08:23:33.999351 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 08:23:33.999729 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:33.990340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:44727 (local address 127.25.127.129:43993)
0619 08:23:33.990518 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:33.990554 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:33.991086 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:33.991306 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:33.991309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:33.991971 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:33.992228 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:33.992233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:33.992894 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:33.992895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:33.993767 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:33.993769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:33.994119 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:33.994119 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:33.994218 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:33.994686 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:33.994694 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:33.996135 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:33.998024 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:33.998032 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:33.998034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:33.998381 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:33.998592 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:33.998593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:33.998593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:33.998735 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:33.998966 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:33.998971 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:33.999143 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 08:23:33.999264 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 08:23:33.999438 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":97,"threads_started":1}
I20260619 08:23:34.000299 26353 heartbeater.cc:347] Connected to a master server at 127.25.127.190:44727
I20260619 08:23:34.000393 26353 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:34.000595 26353 heartbeater.cc:511] Master 127.25.127.190:44727 requested a full tablet report, sending...
I20260619 08:23:34.001137 26162 ts_manager.cc:194] Registered new tserver with Master: 1b1747bb9e4d4a5fb3e89d816ec4136b (127.25.127.129:42665)
I20260619 08:23:34.001718 26162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:43993
WARNING: no policy specified for kudu/127.25.127.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
I20260619 08:23:34.020331 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:44727
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:42515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:34.110940 26358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:34.111168 26358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:34.111192 26358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:34.112382 26358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:34.112422 26358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:34.112449 26358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:23:34.113940 26358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:44727
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:34.114189 26358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:34.114419 26358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:34.119697 26363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:34.119686 26366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:34.119659 26364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:34.119891 26358 server_base.cc:1034] running on GCE node
I20260619 08:23:34.120141 26358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:34.120424 26358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:34.121573 26358 hybrid_clock.cc:648] HybridClock initialized: now 1781857414121564 us; error 28 us; skew 500 ppm
Jun 19 08:23:34 dist-test-slave-5rhg krb5kdc[26117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857414, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:34.124087 26358 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:23:34.124575 26358 webserver.cc:460] Webserver started at http://127.25.127.130:35073/ using document root <none> and password file <none>
I20260619 08:23:34.124763 26358 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:34.124802 26358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:34.124897 26358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:34.125986 26358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "ba2f885cd89f474eacc93584bd8d019b"
format_stamp: "Formatted at 2026-06-19 08:23:34 on dist-test-slave-5rhg"
server_key: "550da1ab39cad31b5e1601cb313759ed"
server_key_iv: "b07a2ea0d8b59d8bf8b937fb0adc99b2"
server_key_version: "encryptionkey@0"
I20260619 08:23:34.126303 26358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "ba2f885cd89f474eacc93584bd8d019b"
format_stamp: "Formatted at 2026-06-19 08:23:34 on dist-test-slave-5rhg"
server_key: "550da1ab39cad31b5e1601cb313759ed"
server_key_iv: "b07a2ea0d8b59d8bf8b937fb0adc99b2"
server_key_version: "encryptionkey@0"
I20260619 08:23:34.127625 26358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:23:34.128425 26373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:34.128593 26358 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:34.128677 26358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "ba2f885cd89f474eacc93584bd8d019b"
format_stamp: "Formatted at 2026-06-19 08:23:34 on dist-test-slave-5rhg"
server_key: "550da1ab39cad31b5e1601cb313759ed"
server_key_iv: "b07a2ea0d8b59d8bf8b937fb0adc99b2"
server_key_version: "encryptionkey@0"
I20260619 08:23:34.128733 26358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:34.152798 26358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:34.155264 26358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:34.155380 26358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:34.155583 26358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:34.155921 26358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:34.155956 26358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:34.155982 26358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:34.156031 26358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:34.162007 26358 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:36283
I20260619 08:23:34.162135 26486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:36283 every 8 connection(s)
I20260619 08:23:34.162426 26358 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:23:34.165073 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 26358
I20260619 08:23:34.165149 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:23:34.165318 26110 external_mini_cluster.cc:1428] Setting key 7f278b8113e0f931743c2be11b1d73c7
Jun 19 08:23:34 dist-test-slave-5rhg krb5kdc[26117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857414, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:34.172809 26200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:34.163542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:54453 (local address 127.25.127.190:44727)
0619 08:23:34.163594 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:34.163596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:34.163607 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:23:34.164329 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:34.164330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:34.164340 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:34.164381 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:34.165372 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:34.166115 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:34.167141 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:34.167341 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:34.167434 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:34.169730 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:34.169731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:34.169732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:34.169739 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:34.171334 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:34.171746 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:34.171747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:34.171748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:34.171808 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:34.172071 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:34.172072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:34.172073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:34.172251 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:34.172327 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:34.172614 (+ 287us) server_negotiation.cc:299] Negotiation successful
0619 08:23:34.172713 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:23:34.173017 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:34.163247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:44727 (local address 127.25.127.130:54453)
0619 08:23:34.163450 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:34.163505 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:34.164225 (+ 720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:34.164446 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:34.164451 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:34.165019 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:34.165278 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:34.165284 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:34.166205 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:34.166206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:34.167026 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:34.167029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:34.167439 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:34.167440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:34.167578 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:34.168196 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:34.168206 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:34.169631 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:34.171417 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:34.171424 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:34.171433 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:34.171656 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:34.171882 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:34.171883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:34.171883 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:34.171994 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:34.172318 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:34.172325 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:34.172477 (+ 152us) client_negotiation.cc:769] Sending connection context
0619 08:23:34.172594 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 08:23:34.172763 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.25.127.131@KRBTEST.COM; defaulting to no policy
I20260619 08:23:34.173492 26487 heartbeater.cc:347] Connected to a master server at 127.25.127.190:44727
I20260619 08:23:34.173609 26487 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:34.173801 26487 heartbeater.cc:511] Master 127.25.127.190:44727 requested a full tablet report, sending...
I20260619 08:23:34.174216 26162 ts_manager.cc:194] Registered new tserver with Master: ba2f885cd89f474eacc93584bd8d019b (127.25.127.130:36283)
I20260619 08:23:34.174654 26162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:54453
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
I20260619 08:23:34.190212 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:44727
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:42515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:34.274813 26492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:34.275023 26492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:34.275056 26492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:34.276285 26492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:34.276325 26492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:34.276391 26492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:23:34.278043 26492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:44727
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:34.278301 26492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:34.278544 26492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:34.283710 26498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:34.283807 26492 server_base.cc:1034] running on GCE node
W20260619 08:23:34.283751 26497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:34.283710 26500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:34.284243 26492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:34.284528 26492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:34.285683 26492 hybrid_clock.cc:648] HybridClock initialized: now 1781857414285676 us; error 33 us; skew 500 ppm
Jun 19 08:23:34 dist-test-slave-5rhg krb5kdc[26117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857414, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:34.288457 26492 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:23:34.288908 26492 webserver.cc:460] Webserver started at http://127.25.127.131:43745/ using document root <none> and password file <none>
I20260619 08:23:34.289151 26492 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:34.289217 26492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:34.289361 26492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:34.290441 26492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "de1d306a378541898f951d66847d7090"
format_stamp: "Formatted at 2026-06-19 08:23:34 on dist-test-slave-5rhg"
server_key: "268e81de5ee93c87ace0c727ac9f6398"
server_key_iv: "ee07b5c775670e720133c737794de641"
server_key_version: "encryptionkey@0"
I20260619 08:23:34.290839 26492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "de1d306a378541898f951d66847d7090"
format_stamp: "Formatted at 2026-06-19 08:23:34 on dist-test-slave-5rhg"
server_key: "268e81de5ee93c87ace0c727ac9f6398"
server_key_iv: "ee07b5c775670e720133c737794de641"
server_key_version: "encryptionkey@0"
I20260619 08:23:34.292335 26492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260619 08:23:34.293165 26507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:34.293359 26492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:23:34.293490 26492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "de1d306a378541898f951d66847d7090"
format_stamp: "Formatted at 2026-06-19 08:23:34 on dist-test-slave-5rhg"
server_key: "268e81de5ee93c87ace0c727ac9f6398"
server_key_iv: "ee07b5c775670e720133c737794de641"
server_key_version: "encryptionkey@0"
I20260619 08:23:34.293574 26492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:34.315374 26492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:34.317759 26492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:34.317931 26492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:34.318176 26492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:34.318490 26492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:34.318543 26492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:34.318585 26492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:34.318617 26492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:34.325973 26492 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:39953
I20260619 08:23:34.326015 26620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:39953 every 8 connection(s)
I20260619 08:23:34.326346 26492 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:23:34 dist-test-slave-5rhg krb5kdc[26117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857414, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:34.335438 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 26492
I20260619 08:23:34.335510 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:23:34.335669 26110 external_mini_cluster.cc:1428] Setting key 0ca4abf474c316ad86caed0d86b549b2
I20260619 08:23:34.336496 26200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:34.327052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:46157 (local address 127.25.127.190:44727)
0619 08:23:34.327103 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:34.327105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:34.327361 (+ 256us) server_negotiation.cc:407] Connection header received
0619 08:23:34.328420 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:34.328421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:34.328432 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:34.328500 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:34.329098 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:34.329609 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:34.330561 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:34.330745 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:34.330795 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:34.333468 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:34.333470 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:34.333470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:34.333484 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:34.334755 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:34.335197 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:34.335198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:34.335199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:34.335258 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:34.335705 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:34.335706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:34.335706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:34.335873 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:34.335965 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:34.336311 (+ 346us) server_negotiation.cc:299] Negotiation successful
0619 08:23:34.336396 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:23:34.336699 26623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:34.327017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:44727 (local address 127.25.127.131:46157)
0619 08:23:34.327213 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:34.327256 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:34.328327 (+ 1071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:34.328563 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:34.328566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:34.328825 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:34.329027 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:34.329031 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:34.329688 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:34.329688 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:34.330491 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:34.330493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:34.330838 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:34.330839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:34.330961 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:34.331559 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:34.331568 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:34.333328 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:34.334843 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:34.334851 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:34.334854 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:34.335116 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:34.335466 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:34.335467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:34.335467 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:34.335628 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:34.335956 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:34.335963 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:34.336139 (+ 176us) client_negotiation.cc:769] Sending connection context
0619 08:23:34.336293 (+ 154us) client_negotiation.cc:240] Negotiation successful
0619 08:23:34.336473 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260619 08:23:34.337232 26621 heartbeater.cc:347] Connected to a master server at 127.25.127.190:44727
I20260619 08:23:34.337316 26621 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:34.337461 26621 heartbeater.cc:511] Master 127.25.127.190:44727 requested a full tablet report, sending...
I20260619 08:23:34.337755 26162 ts_manager.cc:194] Registered new tserver with Master: de1d306a378541898f951d66847d7090 (127.25.127.131:39953)
I20260619 08:23:34.338157 26162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:46157
I20260619 08:23:34.347998 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 08:23:34 dist-test-slave-5rhg krb5kdc[26117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857414, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 08:23:34 dist-test-slave-5rhg krb5kdc[26117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857414, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.127.129@KRBTEST.COM
I20260619 08:23:34.371227 26628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:34.363407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46334 (local address 127.25.127.129:42665)
0619 08:23:34.363597 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:34.363602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:34.363613 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:23:34.363643 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:34.363645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:34.363660 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:34.363728 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:34.364329 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:34.365103 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:34.365801 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:34.366128 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:34.366182 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:34.368406 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:34.368415 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:34.368415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:34.368422 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:34.369998 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:34.370307 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:34.370310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:34.370310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:34.370377 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:34.370583 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:34.370584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:34.370584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:34.370721 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:34.370830 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:34.370901 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 08:23:34.371047 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":103,"threads_started":1}
W20260619 08:23:34.371610 26267 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46334
Jun 19 08:23:34 dist-test-slave-5rhg krb5kdc[26117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 08:23:34 dist-test-slave-5rhg krb5kdc[26117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.129@KRBTEST.COM
I20260619 08:23:34.393662 26628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:34.386932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46340 (local address 127.25.127.129:42665)
0619 08:23:34.387024 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:34.387026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:34.387076 (+ 50us) server_negotiation.cc:407] Connection header received
0619 08:23:34.387126 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:34.387127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:34.387136 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:34.387186 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:34.387727 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:34.388341 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:34.389059 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:34.389258 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:34.389310 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:34.391513 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:34.391514 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:34.391514 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:34.391521 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:34.392753 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:34.393064 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:34.393065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:34.393065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:34.393112 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:34.393274 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:34.393275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:34.393275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:34.393361 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:34.393503 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:34.393529 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 08:23:34.393595 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 08:23:34.394210 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 26224
I20260619 08:23:34.399267 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 26358
I20260619 08:23:34.403542 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 26492
I20260619 08:23:34.408064 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 26130
2026-06-19T08:23:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1910 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:23:34 dist-test-slave-5rhg krb5kdc[26635](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:23:34 dist-test-slave-5rhg krb5kdc[26635](info): set up 2 sockets
Jun 19 08:23:34 dist-test-slave-5rhg krb5kdc[26635](info): commencing operation
krb5kdc: starting...
W20260619 08:23:35.440459 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:23:35 dist-test-slave-5rhg krb5kdc[26635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:23:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:23:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:23:35.553315 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:38735
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:43239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:38735
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:35.642226 26648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:35.642436 26648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:35.642458 26648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:35.643539 26648 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:23:35.643576 26648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:35.643590 26648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:35.643603 26648 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:23:35.643615 26648 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:23:35.645047 26648 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:43239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:38735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:38735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:35.645244 26648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:35.645421 26648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:35.650381 26656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:35.650537 26648 server_base.cc:1034] running on GCE node
W20260619 08:23:35.650388 26654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:35.650388 26653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:35.650879 26648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:35.651232 26648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:35.652392 26648 hybrid_clock.cc:648] HybridClock initialized: now 1781857415652377 us; error 48 us; skew 500 ppm
Jun 19 08:23:35 dist-test-slave-5rhg krb5kdc[26635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857415, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:35.655098 26648 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:23:35.655545 26648 webserver.cc:460] Webserver started at http://127.25.127.190:33077/ using document root <none> and password file <none>
I20260619 08:23:35.655794 26648 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:35.655865 26648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:35.656013 26648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:35.657181 26648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "83eaf6cc1f654970986d67634a77fcdb"
format_stamp: "Formatted at 2026-06-19 08:23:35 on dist-test-slave-5rhg"
server_key: "e79dda83626aa13dc5b8c67c512b87c0"
server_key_iv: "b0b617b8e9b9cdebcdf45b34e3b0974f"
server_key_version: "encryptionkey@0"
I20260619 08:23:35.657548 26648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "83eaf6cc1f654970986d67634a77fcdb"
format_stamp: "Formatted at 2026-06-19 08:23:35 on dist-test-slave-5rhg"
server_key: "e79dda83626aa13dc5b8c67c512b87c0"
server_key_iv: "b0b617b8e9b9cdebcdf45b34e3b0974f"
server_key_version: "encryptionkey@0"
I20260619 08:23:35.658924 26648 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:23:35.659672 26663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:35.659870 26648 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:23:35.659974 26648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "83eaf6cc1f654970986d67634a77fcdb"
format_stamp: "Formatted at 2026-06-19 08:23:35 on dist-test-slave-5rhg"
server_key: "e79dda83626aa13dc5b8c67c512b87c0"
server_key_iv: "b0b617b8e9b9cdebcdf45b34e3b0974f"
server_key_version: "encryptionkey@0"
I20260619 08:23:35.660082 26648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:35.690554 26648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:35.693030 26648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:35.693223 26648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:35.697300 26648 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:38735
I20260619 08:23:35.697333 26715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:38735 every 8 connection(s)
I20260619 08:23:35.697714 26648 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:23:35.698598 26716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:35.701267 26716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb: Bootstrap starting.
I20260619 08:23:35.701876 26716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:35.702175 26716 log.cc:824] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:35.702869 26716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb: No bootstrap required, opened a new log
I20260619 08:23:35.704582 26716 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eaf6cc1f654970986d67634a77fcdb" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38735 } }
I20260619 08:23:35.704715 26716 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:35.704736 26716 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83eaf6cc1f654970986d67634a77fcdb, State: Initialized, Role: FOLLOWER
I20260619 08:23:35.704909 26716 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eaf6cc1f654970986d67634a77fcdb" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38735 } }
I20260619 08:23:35.704972 26716 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:23:35.705014 26716 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:23:35.705063 26716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:35.705439 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 26648
I20260619 08:23:35.705554 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:23:35.705621 26716 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eaf6cc1f654970986d67634a77fcdb" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38735 } }
I20260619 08:23:35.705739 26110 external_mini_cluster.cc:1428] Setting key cdb7f0a948408b17ef92ec567b01adea
I20260619 08:23:35.705762 26716 leader_election.cc:302] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83eaf6cc1f654970986d67634a77fcdb; no voters:
I20260619 08:23:35.705950 26716 leader_election.cc:288] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:23:35.706022 26719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:35.706281 26716 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:35.706209 26719 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb [term 1 LEADER]: Becoming Leader. State: Replica: 83eaf6cc1f654970986d67634a77fcdb, State: Running, Role: LEADER
I20260619 08:23:35.706472 26719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eaf6cc1f654970986d67634a77fcdb" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38735 } }
I20260619 08:23:35.706974 26721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83eaf6cc1f654970986d67634a77fcdb. Latest consensus state: current_term: 1 leader_uuid: "83eaf6cc1f654970986d67634a77fcdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eaf6cc1f654970986d67634a77fcdb" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38735 } } }
I20260619 08:23:35.707072 26721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:35.707386 26720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83eaf6cc1f654970986d67634a77fcdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eaf6cc1f654970986d67634a77fcdb" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38735 } } }
I20260619 08:23:35.707461 26720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:35.707442 26730 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:23:35.707940 26730 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:23:35.710091 26730 catalog_manager.cc:1347] Generated new cluster ID: 75197ec3fa0d4b16a27b219d344d2872
I20260619 08:23:35.710147 26730 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:23:35 dist-test-slave-5rhg krb5kdc[26635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:35.716882 26723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:35.706302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34950 (local address 127.25.127.190:38735)
0619 08:23:35.706677 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:35.706687 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:35.706723 (+ 36us) server_negotiation.cc:407] Connection header received
0619 08:23:35.706849 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:35.706859 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:35.706919 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:35.708120 (+ 1201us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:35.708246 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:35.709658 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:35.710544 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:35.710977 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:35.711130 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:35.713522 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:35.713526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:35.713526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:35.713536 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:35.715224 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:35.715591 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:35.715594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:35.715595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:35.715692 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:35.715922 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:35.715922 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:35.715923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:35.716217 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:35.716390 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:35.716486 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 08:23:35.716722 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":62,"threads_started":1}
I20260619 08:23:35.734664 26730 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:23:35.735217 26730 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:23:35.746824 26730 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83eaf6cc1f654970986d67634a77fcdb: Generated new TSK 0
I20260619 08:23:35.747062 26730 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
I20260619 08:23:35.776971 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:38735
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:43239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:35.861361 26743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:35.861625 26743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:35.861649 26743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:35.862924 26743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:35.862963 26743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:35.863010 26743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:23:35.864563 26743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:43239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:38735
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:35.864797 26743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:35.865020 26743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:23:35.869985 26743 server_base.cc:1034] running on GCE node
W20260619 08:23:35.869966 26749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:35.869963 26748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:35.870136 26751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:35.870373 26743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:35.870612 26743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:35.871755 26743 hybrid_clock.cc:648] HybridClock initialized: now 1781857415871743 us; error 29 us; skew 500 ppm
Jun 19 08:23:35 dist-test-slave-5rhg krb5kdc[26635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857415, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:35.874250 26743 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:23:35.874686 26743 webserver.cc:460] Webserver started at http://127.25.127.129:33149/ using document root <none> and password file <none>
I20260619 08:23:35.874900 26743 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:35.874969 26743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:35.875114 26743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:35.876142 26743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "206c75ab6f0641c1be4208061af14acc"
format_stamp: "Formatted at 2026-06-19 08:23:35 on dist-test-slave-5rhg"
server_key: "aaa3efcb6f9f3b7fd6d7d31885fc4cea"
server_key_iv: "5a82a5b758e74d6fa6076e3db2936397"
server_key_version: "encryptionkey@0"
I20260619 08:23:35.876566 26743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "206c75ab6f0641c1be4208061af14acc"
format_stamp: "Formatted at 2026-06-19 08:23:35 on dist-test-slave-5rhg"
server_key: "aaa3efcb6f9f3b7fd6d7d31885fc4cea"
server_key_iv: "5a82a5b758e74d6fa6076e3db2936397"
server_key_version: "encryptionkey@0"
I20260619 08:23:35.877820 26743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:35.878526 26758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:35.878696 26743 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:35.878791 26743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "206c75ab6f0641c1be4208061af14acc"
format_stamp: "Formatted at 2026-06-19 08:23:35 on dist-test-slave-5rhg"
server_key: "aaa3efcb6f9f3b7fd6d7d31885fc4cea"
server_key_iv: "5a82a5b758e74d6fa6076e3db2936397"
server_key_version: "encryptionkey@0"
I20260619 08:23:35.878876 26743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:35.890870 26743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:35.892925 26743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:35.893083 26743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:35.893313 26743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:35.893636 26743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:35.893692 26743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:35.893734 26743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:35.893774 26743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:35.899592 26743 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:45879
I20260619 08:23:35.899646 26871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:45879 every 8 connection(s)
I20260619 08:23:35.899986 26743 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:23:35.902076 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 26743
I20260619 08:23:35.902180 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:23:35.902333 26110 external_mini_cluster.cc:1428] Setting key 8089c5e145b51155fcfdf932afd666c0
Jun 19 08:23:35 dist-test-slave-5rhg krb5kdc[26635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857415, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.127.130@KRBTEST.COM; defaulting to no policy
I20260619 08:23:35.910719 26723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:35.900760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:43033 (local address 127.25.127.190:38735)
0619 08:23:35.900808 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:35.900810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:35.900993 (+ 183us) server_negotiation.cc:407] Connection header received
0619 08:23:35.901962 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:35.901964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:35.901983 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:35.902036 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:35.902888 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:35.903649 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:35.905050 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:35.905244 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:35.905296 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:35.907867 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:35.907869 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:35.907869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:35.907877 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:35.909300 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:35.909727 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:35.909728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:35.909729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:35.909792 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:35.910058 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:35.910059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:35.910059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:35.910175 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:35.910253 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:35.910540 (+ 287us) server_negotiation.cc:299] Negotiation successful
0619 08:23:35.910629 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:23:35.910919 26874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:35.900720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:38735 (local address 127.25.127.129:43033)
0619 08:23:35.900854 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:35.900901 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:35.901829 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:35.902107 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:35.902111 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:35.902488 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:35.902783 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:35.902789 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:35.903744 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:35.903745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:35.904944 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:35.904956 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:35.905324 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:35.905325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:35.905470 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:35.905968 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:35.905979 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:35.907777 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:35.909388 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:35.909396 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:35.909398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:35.909632 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:35.909864 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:35.909865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:35.909865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:35.909983 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:35.910242 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:35.910249 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:35.910396 (+ 147us) client_negotiation.cc:769] Sending connection context
0619 08:23:35.910504 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 08:23:35.910668 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":61,"threads_started":1}
I20260619 08:23:35.911396 26872 heartbeater.cc:347] Connected to a master server at 127.25.127.190:38735
I20260619 08:23:35.911501 26872 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:35.911684 26872 heartbeater.cc:511] Master 127.25.127.190:38735 requested a full tablet report, sending...
I20260619 08:23:35.912292 26680 ts_manager.cc:194] Registered new tserver with Master: 206c75ab6f0641c1be4208061af14acc (127.25.127.129:45879)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.130@KRBTEST.COM" created.
I20260619 08:23:35.913062 26680 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:43033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
I20260619 08:23:35.924891 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:38735
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:43239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:36.005200 26877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:36.005497 26877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:36.005537 26877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:36.006901 26877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:36.006943 26877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:36.006973 26877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:23:36.008561 26877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:43239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:38735
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:36.008792 26877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:36.009037 26877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:36.013880 26885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:36.013906 26882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:36.013923 26883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:36.014106 26877 server_base.cc:1034] running on GCE node
I20260619 08:23:36.014319 26877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:36.014583 26877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:36.015748 26877 hybrid_clock.cc:648] HybridClock initialized: now 1781857416015736 us; error 29 us; skew 500 ppm
Jun 19 08:23:36 dist-test-slave-5rhg krb5kdc[26635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857416, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:36.019241 26877 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:23:36.019696 26877 webserver.cc:460] Webserver started at http://127.25.127.130:40247/ using document root <none> and password file <none>
I20260619 08:23:36.019969 26877 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:36.020013 26877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:36.020182 26877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:36.021754 26877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "f61a562956414c089e732a6a2909d4d4"
format_stamp: "Formatted at 2026-06-19 08:23:36 on dist-test-slave-5rhg"
server_key: "3dd02ca1c695ebb04491f4cec3368f85"
server_key_iv: "5aa70186e5bac43d79dcb7bd924c3366"
server_key_version: "encryptionkey@0"
I20260619 08:23:36.022169 26877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "f61a562956414c089e732a6a2909d4d4"
format_stamp: "Formatted at 2026-06-19 08:23:36 on dist-test-slave-5rhg"
server_key: "3dd02ca1c695ebb04491f4cec3368f85"
server_key_iv: "5aa70186e5bac43d79dcb7bd924c3366"
server_key_version: "encryptionkey@0"
I20260619 08:23:36.024071 26877 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260619 08:23:36.025012 26892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:36.025197 26877 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:36.025297 26877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "f61a562956414c089e732a6a2909d4d4"
format_stamp: "Formatted at 2026-06-19 08:23:36 on dist-test-slave-5rhg"
server_key: "3dd02ca1c695ebb04491f4cec3368f85"
server_key_iv: "5aa70186e5bac43d79dcb7bd924c3366"
server_key_version: "encryptionkey@0"
I20260619 08:23:36.025382 26877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:36.044992 26877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:36.048048 26877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:36.048233 26877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:36.048508 26877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:36.048895 26877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:36.048957 26877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:36.048995 26877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:36.049021 26877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:36.056476 26877 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:40083
I20260619 08:23:36.056519 27005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:40083 every 8 connection(s)
I20260619 08:23:36.056902 26877 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:23:36.059445 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 26877
I20260619 08:23:36.059544 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:23:36.059729 26110 external_mini_cluster.cc:1428] Setting key 17fa068becbfc19a6ebbdee4e91ca5af
Jun 19 08:23:36 dist-test-slave-5rhg krb5kdc[26635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857416, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.127.131@KRBTEST.COM; defaulting to no policy
I20260619 08:23:36.069639 26723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:36.057693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:44277 (local address 127.25.127.190:38735)
0619 08:23:36.057745 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:36.057748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:36.058007 (+ 259us) server_negotiation.cc:407] Connection header received
0619 08:23:36.059445 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:36.059447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:36.059462 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:36.059535 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:36.060502 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.061248 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.062999 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.063350 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.063414 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:36.066373 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:36.066376 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:36.066376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:36.066384 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:36.068139 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:36.068659 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:36.068661 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:36.068661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:36.068720 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:36.069018 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:36.069019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:36.069019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:36.069124 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:36.069176 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:36.069497 (+ 321us) server_negotiation.cc:299] Negotiation successful
0619 08:23:36.069572 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:23:36.069995 27008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:36.057687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:38735 (local address 127.25.127.130:44277)
0619 08:23:36.057856 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:36.057909 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:36.059300 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:36.059676 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:36.059682 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:36.060059 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:36.060366 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:36.060373 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.061365 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.061366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:36.062853 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:36.062856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.063459 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.063461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:36.063616 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:36.064241 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:36.064251 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:36.066250 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:36.068255 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:36.068263 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:36.068265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:36.068568 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:36.068786 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:36.068787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:36.068788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:36.068935 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:36.069190 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:36.069197 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:36.069362 (+ 165us) client_negotiation.cc:769] Sending connection context
0619 08:23:36.069480 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 08:23:36.069735 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260619 08:23:36.070513 27006 heartbeater.cc:347] Connected to a master server at 127.25.127.190:38735
I20260619 08:23:36.070608 27006 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:36.070752 27006 heartbeater.cc:511] Master 127.25.127.190:38735 requested a full tablet report, sending...
I20260619 08:23:36.071108 26680 ts_manager.cc:194] Registered new tserver with Master: f61a562956414c089e732a6a2909d4d4 (127.25.127.130:40083)
I20260619 08:23:36.071455 26680 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:44277
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
I20260619 08:23:36.084935 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:38735
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:43239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:36.166375 27011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:36.166613 27011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:36.166639 27011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:36.167865 27011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:36.167908 27011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:36.167935 27011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:23:36.169431 27011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:43239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:38735
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:36.169697 27011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:36.169945 27011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:36.174856 27016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:36.174892 27017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:36.174856 27019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:36.175227 27011 server_base.cc:1034] running on GCE node
I20260619 08:23:36.175432 27011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:36.175622 27011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:36.176753 27011 hybrid_clock.cc:648] HybridClock initialized: now 1781857416176734 us; error 34 us; skew 500 ppm
Jun 19 08:23:36 dist-test-slave-5rhg krb5kdc[26635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857416, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:36.179227 27011 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:23:36.179668 27011 webserver.cc:460] Webserver started at http://127.25.127.131:39625/ using document root <none> and password file <none>
I20260619 08:23:36.179889 27011 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:36.179932 27011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:36.180087 27011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:36.181216 27011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "a277c8e2370f46db93da8debc15dad39"
format_stamp: "Formatted at 2026-06-19 08:23:36 on dist-test-slave-5rhg"
server_key: "7580754197574797dbca332beb4018aa"
server_key_iv: "d4bff0a26ab717c836d340145f97a634"
server_key_version: "encryptionkey@0"
I20260619 08:23:36.181569 27011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "a277c8e2370f46db93da8debc15dad39"
format_stamp: "Formatted at 2026-06-19 08:23:36 on dist-test-slave-5rhg"
server_key: "7580754197574797dbca332beb4018aa"
server_key_iv: "d4bff0a26ab717c836d340145f97a634"
server_key_version: "encryptionkey@0"
I20260619 08:23:36.182899 27011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:23:36.183632 27026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:36.183805 27011 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:36.183903 27011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "a277c8e2370f46db93da8debc15dad39"
format_stamp: "Formatted at 2026-06-19 08:23:36 on dist-test-slave-5rhg"
server_key: "7580754197574797dbca332beb4018aa"
server_key_iv: "d4bff0a26ab717c836d340145f97a634"
server_key_version: "encryptionkey@0"
I20260619 08:23:36.183990 27011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:36.220137 27011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:36.222265 27011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:36.222427 27011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:36.222667 27011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:36.222985 27011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:36.223048 27011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:36.223090 27011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:36.223119 27011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:36.229178 27011 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:42141
I20260619 08:23:36.229223 27139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:42141 every 8 connection(s)
I20260619 08:23:36.229543 27011 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:23:36 dist-test-slave-5rhg krb5kdc[26635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857416, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:36.238875 26723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:36.230254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:42725 (local address 127.25.127.190:38735)
0619 08:23:36.230330 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:36.230332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:36.230541 (+ 209us) server_negotiation.cc:407] Connection header received
0619 08:23:36.231455 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:36.231456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:36.231466 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:36.231540 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:36.232266 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.232770 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.233770 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.233946 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.233996 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:36.236300 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:36.236302 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:36.236302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:36.236309 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:36.237483 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:36.237909 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:36.237910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:36.237910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:36.237970 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:36.238231 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:36.238232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:36.238232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:36.238352 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:36.238427 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:36.238703 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 08:23:36.238782 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 08:23:36.239084 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:36.230250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:38735 (local address 127.25.127.131:42725)
0619 08:23:36.230413 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:36.230450 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:36.231366 (+ 916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:36.231620 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:36.231623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:36.231913 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:36.232107 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:36.232112 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.232849 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.232850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:36.233692 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:36.233695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.234041 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.234042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:36.234158 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:36.234721 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:36.234729 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:36.236140 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:36.237586 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:36.237594 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:36.237596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:36.237821 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:36.238050 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:36.238051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:36.238051 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:36.238163 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:36.238419 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:36.238426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:36.238571 (+ 145us) client_negotiation.cc:769] Sending connection context
0619 08:23:36.238680 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 08:23:36.238842 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260619 08:23:36.239568 27140 heartbeater.cc:347] Connected to a master server at 127.25.127.190:38735
I20260619 08:23:36.239678 27140 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:36.239738 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 27011
I20260619 08:23:36.239849 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:23:36.239888 27140 heartbeater.cc:511] Master 127.25.127.190:38735 requested a full tablet report, sending...
I20260619 08:23:36.240021 26110 external_mini_cluster.cc:1428] Setting key 5faa5f6bbd7d6dbdf1e01901c16a3280
I20260619 08:23:36.240326 26680 ts_manager.cc:194] Registered new tserver with Master: a277c8e2370f46db93da8debc15dad39 (127.25.127.131:42141)
I20260619 08:23:36.240846 26680 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:42725
I20260619 08:23:36.240855 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:23:36.247933 26723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:36.242346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34956 (local address 127.25.127.190:38735)
0619 08:23:36.242407 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:36.242409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:36.242508 (+ 99us) server_negotiation.cc:407] Connection header received
0619 08:23:36.242566 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:36.242567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:36.242577 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:36.242631 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:36.243178 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.243647 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.244433 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.244629 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.244711 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:36.245792 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:36.245793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:36.245793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:36.245800 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:36.246953 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:36.247268 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:36.247268 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:36.247269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:36.247315 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:36.247531 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:36.247531 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:36.247531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:36.247658 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:36.247730 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:36.247823 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 08:23:36.247876 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 08:23:36.250015 26680 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34956:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:23:36.251556 26680 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:23:36.258657 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:36.253922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:40083 (local address 127.0.0.1:41434)
0619 08:23:36.254440 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:36.254450 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:36.254512 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:36.254845 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:36.254845 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:36.254847 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:36.254923 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:36.254926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.256245 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.256246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:36.257358 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:36.257361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.258336 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.258337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:36.258421 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:36.258439 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 08:23:36.258514 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 08:23:36.258578 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":49,"thread_start_us":78,"threads_started":1}
I20260619 08:23:36.258679 27153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:36.254022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60502 (local address 127.25.127.131:42141)
0619 08:23:36.254452 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:36.254456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:36.254465 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:23:36.254532 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:36.254535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:36.254552 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:36.254656 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:36.254897 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.256033 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.257430 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.258170 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.258244 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:36.258298 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:36.258563 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 08:23:36.258605 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":77,"threads_started":1}
I20260619 08:23:36.258850 27152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:36.253895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:42141 (local address 127.0.0.1:60502)
0619 08:23:36.254057 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:36.254105 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:36.254173 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:36.254722 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:36.254726 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:36.254728 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:36.254803 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:36.254805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.256200 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.256201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:36.257352 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:36.257355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.258337 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.258337 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:36.258421 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:36.258439 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 08:23:36.258518 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 08:23:36.258802 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":274,"thread_start_us":46,"threads_started":1}
I20260619 08:23:36.258687 27155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:36.254212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41434 (local address 127.25.127.130:40083)
0619 08:23:36.254390 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:36.254395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:36.254514 (+ 119us) server_negotiation.cc:407] Connection header received
0619 08:23:36.254561 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:36.254563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:36.254581 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:36.254758 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:36.255070 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.256127 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.257479 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.258170 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.258244 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:36.258298 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:36.258550 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 08:23:36.258605 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
I20260619 08:23:36.259665 27074 tablet_service.cc:1467] Processing CreateTablet for tablet 144ae2ba886e46b3a981022b841dc2fd (DEFAULT_TABLE table=test-table [id=f08946893a4d4a6998f22da2605026af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:36.260008 27074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 144ae2ba886e46b3a981022b841dc2fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:36.259902 26940 tablet_service.cc:1467] Processing CreateTablet for tablet 144ae2ba886e46b3a981022b841dc2fd (DEFAULT_TABLE table=test-table [id=f08946893a4d4a6998f22da2605026af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:36.260275 26940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 144ae2ba886e46b3a981022b841dc2fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:36.262830 27157 tablet_bootstrap.cc:492] T 144ae2ba886e46b3a981022b841dc2fd P a277c8e2370f46db93da8debc15dad39: Bootstrap starting.
I20260619 08:23:36.262847 27158 tablet_bootstrap.cc:492] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4: Bootstrap starting.
I20260619 08:23:36.263512 27157 tablet_bootstrap.cc:654] T 144ae2ba886e46b3a981022b841dc2fd P a277c8e2370f46db93da8debc15dad39: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:36.263550 27158 tablet_bootstrap.cc:654] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:36.264032 27157 log.cc:824] T 144ae2ba886e46b3a981022b841dc2fd P a277c8e2370f46db93da8debc15dad39: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:36.264101 27158 log.cc:824] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:36.264612 27151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:36.254103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:45879 (local address 127.0.0.1:38660)
0619 08:23:36.254769 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:36.254782 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:36.254834 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:36.256252 (+ 1418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:36.256253 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:36.256255 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:36.256347 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:36.256349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.258215 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.258216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:36.263161 (+ 4945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:36.263164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.264329 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.264330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:36.264397 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:36.264401 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:23:36.264493 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 08:23:36.264539 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":19,"thread_start_us":71,"threads_started":1}
I20260619 08:23:36.264636 27156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:36.254244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38660 (local address 127.25.127.129:45879)
0619 08:23:36.254417 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:36.254422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:36.254907 (+ 485us) server_negotiation.cc:407] Connection header received
0619 08:23:36.254962 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:36.254966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:36.254982 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:36.255062 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:36.257446 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.258112 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.263262 (+ 5150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.264122 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.264221 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:36.264288 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:36.264520 (+ 232us) server_negotiation.cc:299] Negotiation successful
0619 08:23:36.264564 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
I20260619 08:23:36.264833 27158 tablet_bootstrap.cc:492] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4: No bootstrap required, opened a new log
I20260619 08:23:36.264843 27157 tablet_bootstrap.cc:492] T 144ae2ba886e46b3a981022b841dc2fd P a277c8e2370f46db93da8debc15dad39: No bootstrap required, opened a new log
I20260619 08:23:36.264899 27157 ts_tablet_manager.cc:1397] T 144ae2ba886e46b3a981022b841dc2fd P a277c8e2370f46db93da8debc15dad39: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:23:36.264899 27158 ts_tablet_manager.cc:1397] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:23:36.265205 26806 tablet_service.cc:1467] Processing CreateTablet for tablet 144ae2ba886e46b3a981022b841dc2fd (DEFAULT_TABLE table=test-table [id=f08946893a4d4a6998f22da2605026af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:36.265550 26806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 144ae2ba886e46b3a981022b841dc2fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:36.266992 27157 raft_consensus.cc:348] T 144ae2ba886e46b3a981022b841dc2fd P a277c8e2370f46db93da8debc15dad39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a277c8e2370f46db93da8debc15dad39" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42141 } } peers { permanent_uuid: "f61a562956414c089e732a6a2909d4d4" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40083 } } peers { permanent_uuid: "206c75ab6f0641c1be4208061af14acc" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45879 } }
I20260619 08:23:36.267043 27158 raft_consensus.cc:348] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a277c8e2370f46db93da8debc15dad39" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42141 } } peers { permanent_uuid: "f61a562956414c089e732a6a2909d4d4" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40083 } } peers { permanent_uuid: "206c75ab6f0641c1be4208061af14acc" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45879 } }
I20260619 08:23:36.267180 27157 raft_consensus.cc:374] T 144ae2ba886e46b3a981022b841dc2fd P a277c8e2370f46db93da8debc15dad39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:36.267262 27158 raft_consensus.cc:374] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:36.267267 27157 raft_consensus.cc:729] T 144ae2ba886e46b3a981022b841dc2fd P a277c8e2370f46db93da8debc15dad39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a277c8e2370f46db93da8debc15dad39, State: Initialized, Role: FOLLOWER
I20260619 08:23:36.267298 27158 raft_consensus.cc:729] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f61a562956414c089e732a6a2909d4d4, State: Initialized, Role: FOLLOWER
I20260619 08:23:36.267403 27157 consensus_queue.cc:260] T 144ae2ba886e46b3a981022b841dc2fd P a277c8e2370f46db93da8debc15dad39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a277c8e2370f46db93da8debc15dad39" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42141 } } peers { permanent_uuid: "f61a562956414c089e732a6a2909d4d4" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40083 } } peers { permanent_uuid: "206c75ab6f0641c1be4208061af14acc" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45879 } }
I20260619 08:23:36.267403 27158 consensus_queue.cc:260] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a277c8e2370f46db93da8debc15dad39" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42141 } } peers { permanent_uuid: "f61a562956414c089e732a6a2909d4d4" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40083 } } peers { permanent_uuid: "206c75ab6f0641c1be4208061af14acc" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45879 } }
I20260619 08:23:36.267750 27140 heartbeater.cc:503] Master 127.25.127.190:38735 was elected leader, sending a full tablet report...
I20260619 08:23:36.267783 27158 ts_tablet_manager.cc:1428] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:23:36.267930 27006 heartbeater.cc:503] Master 127.25.127.190:38735 was elected leader, sending a full tablet report...
I20260619 08:23:36.268538 27163 tablet_bootstrap.cc:492] T 144ae2ba886e46b3a981022b841dc2fd P 206c75ab6f0641c1be4208061af14acc: Bootstrap starting.
I20260619 08:23:36.268582 27157 ts_tablet_manager.cc:1428] T 144ae2ba886e46b3a981022b841dc2fd P a277c8e2370f46db93da8debc15dad39: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 08:23:36.269266 27163 tablet_bootstrap.cc:654] T 144ae2ba886e46b3a981022b841dc2fd P 206c75ab6f0641c1be4208061af14acc: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:36.269824 27163 log.cc:824] T 144ae2ba886e46b3a981022b841dc2fd P 206c75ab6f0641c1be4208061af14acc: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:36.270493 27163 tablet_bootstrap.cc:492] T 144ae2ba886e46b3a981022b841dc2fd P 206c75ab6f0641c1be4208061af14acc: No bootstrap required, opened a new log
I20260619 08:23:36.270563 27163 ts_tablet_manager.cc:1397] T 144ae2ba886e46b3a981022b841dc2fd P 206c75ab6f0641c1be4208061af14acc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:23:36.271947 27163 raft_consensus.cc:348] T 144ae2ba886e46b3a981022b841dc2fd P 206c75ab6f0641c1be4208061af14acc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a277c8e2370f46db93da8debc15dad39" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42141 } } peers { permanent_uuid: "f61a562956414c089e732a6a2909d4d4" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40083 } } peers { permanent_uuid: "206c75ab6f0641c1be4208061af14acc" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45879 } }
I20260619 08:23:36.272140 27163 raft_consensus.cc:374] T 144ae2ba886e46b3a981022b841dc2fd P 206c75ab6f0641c1be4208061af14acc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:36.272235 27163 raft_consensus.cc:729] T 144ae2ba886e46b3a981022b841dc2fd P 206c75ab6f0641c1be4208061af14acc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 206c75ab6f0641c1be4208061af14acc, State: Initialized, Role: FOLLOWER
I20260619 08:23:36.272368 27163 consensus_queue.cc:260] T 144ae2ba886e46b3a981022b841dc2fd P 206c75ab6f0641c1be4208061af14acc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a277c8e2370f46db93da8debc15dad39" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42141 } } peers { permanent_uuid: "f61a562956414c089e732a6a2909d4d4" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40083 } } peers { permanent_uuid: "206c75ab6f0641c1be4208061af14acc" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45879 } }
I20260619 08:23:36.272655 27163 ts_tablet_manager.cc:1428] T 144ae2ba886e46b3a981022b841dc2fd P 206c75ab6f0641c1be4208061af14acc: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 08:23:36.272666 26872 heartbeater.cc:503] Master 127.25.127.190:38735 was elected leader, sending a full tablet report...
W20260619 08:23:36.307652 27007 tablet.cc:2369] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:23:36.359027 27162 raft_consensus.cc:482] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:23:36.359156 27162 raft_consensus.cc:504] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a277c8e2370f46db93da8debc15dad39" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42141 } } peers { permanent_uuid: "f61a562956414c089e732a6a2909d4d4" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40083 } } peers { permanent_uuid: "206c75ab6f0641c1be4208061af14acc" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45879 } }
I20260619 08:23:36.359612 27162 leader_election.cc:288] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a277c8e2370f46db93da8debc15dad39 (127.25.127.131:42141), 206c75ab6f0641c1be4208061af14acc (127.25.127.129:45879)
I20260619 08:23:36.362725 27008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:36.359796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:45879 (local address 127.25.127.130:51131)
0619 08:23:36.359860 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:36.359874 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:36.359926 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:36.360109 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:36.360110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:36.360111 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:36.360253 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:36.360255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.360956 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.360957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:36.361766 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:36.361769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.362436 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.362437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:36.362541 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:36.362546 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:23:36.362596 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 08:23:36.362641 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 08:23:36.362761 27156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:36.359889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:51131 (local address 127.25.127.129:45879)
0619 08:23:36.359939 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:36.359941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:36.359953 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:23:36.360017 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:36.360018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:36.360029 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:36.360098 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:36.360344 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.360880 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.361858 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.362340 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.362398 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:36.362405 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:36.362625 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 08:23:36.362681 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:23:36.362910 27153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:36.359896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:49617 (local address 127.25.127.131:42141)
0619 08:23:36.359947 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:36.359949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:36.360159 (+ 210us) server_negotiation.cc:407] Connection header received
0619 08:23:36.360258 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:36.360259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:36.360270 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:36.360327 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:36.360485 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.360971 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.361975 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.362511 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.362587 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:36.362595 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:36.362801 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 08:23:36.362845 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 08:23:36.363085 27166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:36.359814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:42141 (local address 127.25.127.130:49617)
0619 08:23:36.360077 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:36.360088 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:36.360170 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:36.360344 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:36.360345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:36.360346 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:36.360413 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:36.360415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.361053 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.361053 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:36.361888 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:36.361890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.362598 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.362599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:36.362717 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:36.362722 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:23:36.362778 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 08:23:36.362990 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":48,"thread_start_us":127,"threads_started":1}
I20260619 08:23:36.363332 26826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "144ae2ba886e46b3a981022b841dc2fd" candidate_uuid: "f61a562956414c089e732a6a2909d4d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "206c75ab6f0641c1be4208061af14acc" is_pre_election: true
I20260619 08:23:36.363503 26826 raft_consensus.cc:2454] T 144ae2ba886e46b3a981022b841dc2fd P 206c75ab6f0641c1be4208061af14acc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f61a562956414c089e732a6a2909d4d4 in term 0.
I20260619 08:23:36.363503 27094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "144ae2ba886e46b3a981022b841dc2fd" candidate_uuid: "f61a562956414c089e732a6a2909d4d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a277c8e2370f46db93da8debc15dad39" is_pre_election: true
I20260619 08:23:36.363682 27094 raft_consensus.cc:2454] T 144ae2ba886e46b3a981022b841dc2fd P a277c8e2370f46db93da8debc15dad39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f61a562956414c089e732a6a2909d4d4 in term 0.
I20260619 08:23:36.363732 26896 leader_election.cc:302] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 206c75ab6f0641c1be4208061af14acc, f61a562956414c089e732a6a2909d4d4; no voters:
I20260619 08:23:36.363941 27162 raft_consensus.cc:2781] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:23:36.364005 27162 raft_consensus.cc:482] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:23:36.364035 27162 raft_consensus.cc:3037] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:36.364725 27162 raft_consensus.cc:504] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a277c8e2370f46db93da8debc15dad39" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42141 } } peers { permanent_uuid: "f61a562956414c089e732a6a2909d4d4" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40083 } } peers { permanent_uuid: "206c75ab6f0641c1be4208061af14acc" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45879 } }
I20260619 08:23:36.364856 27162 leader_election.cc:288] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4 [CANDIDATE]: Term 1 election: Requested vote from peers a277c8e2370f46db93da8debc15dad39 (127.25.127.131:42141), 206c75ab6f0641c1be4208061af14acc (127.25.127.129:45879)
I20260619 08:23:36.365001 27094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "144ae2ba886e46b3a981022b841dc2fd" candidate_uuid: "f61a562956414c089e732a6a2909d4d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a277c8e2370f46db93da8debc15dad39"
I20260619 08:23:36.365008 26826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "144ae2ba886e46b3a981022b841dc2fd" candidate_uuid: "f61a562956414c089e732a6a2909d4d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "206c75ab6f0641c1be4208061af14acc"
I20260619 08:23:36.365083 27094 raft_consensus.cc:3037] T 144ae2ba886e46b3a981022b841dc2fd P a277c8e2370f46db93da8debc15dad39 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:36.365085 26826 raft_consensus.cc:3037] T 144ae2ba886e46b3a981022b841dc2fd P 206c75ab6f0641c1be4208061af14acc [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:36.365774 26826 raft_consensus.cc:2454] T 144ae2ba886e46b3a981022b841dc2fd P 206c75ab6f0641c1be4208061af14acc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f61a562956414c089e732a6a2909d4d4 in term 1.
I20260619 08:23:36.365934 26896 leader_election.cc:302] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 206c75ab6f0641c1be4208061af14acc, f61a562956414c089e732a6a2909d4d4; no voters:
I20260619 08:23:36.365964 27094 raft_consensus.cc:2454] T 144ae2ba886e46b3a981022b841dc2fd P a277c8e2370f46db93da8debc15dad39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f61a562956414c089e732a6a2909d4d4 in term 1.
I20260619 08:23:36.366019 27162 raft_consensus.cc:2781] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:36.366197 27162 raft_consensus.cc:686] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4 [term 1 LEADER]: Becoming Leader. State: Replica: f61a562956414c089e732a6a2909d4d4, State: Running, Role: LEADER
I20260619 08:23:36.366288 27162 consensus_queue.cc:237] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a277c8e2370f46db93da8debc15dad39" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42141 } } peers { permanent_uuid: "f61a562956414c089e732a6a2909d4d4" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40083 } } peers { permanent_uuid: "206c75ab6f0641c1be4208061af14acc" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45879 } }
I20260619 08:23:36.367133 26680 catalog_manager.cc:5495] T 144ae2ba886e46b3a981022b841dc2fd P f61a562956414c089e732a6a2909d4d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f61a562956414c089e732a6a2909d4d4 (127.25.127.130). New cstate: current_term: 1 leader_uuid: "f61a562956414c089e732a6a2909d4d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a277c8e2370f46db93da8debc15dad39" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f61a562956414c089e732a6a2909d4d4" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "206c75ab6f0641c1be4208061af14acc" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45879 } health_report { overall_health: UNKNOWN } } }
Jun 19 08:23:36 dist-test-slave-5rhg krb5kdc[26635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.129@KRBTEST.COM
I20260619 08:23:36.376292 27156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:36.369277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38670 (local address 127.25.127.129:45879)
0619 08:23:36.369348 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:36.369350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:36.369434 (+ 84us) server_negotiation.cc:407] Connection header received
0619 08:23:36.369500 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:36.369500 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:36.369510 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:36.369566 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:36.370111 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.370561 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.371285 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.371487 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.371559 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:36.373884 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:36.373893 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:36.373893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:36.373899 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:36.375320 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:36.375644 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:36.375647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:36.375647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:36.375695 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:36.375849 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:36.375850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:36.375850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:36.375965 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:36.376078 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:36.376112 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 08:23:36.376223 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Jun 19 08:23:36 dist-test-slave-5rhg krb5kdc[26635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857416, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 08:23:36 dist-test-slave-5rhg krb5kdc[26635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857416, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.127.129@KRBTEST.COM
I20260619 08:23:36.399274 27156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:36.391965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38686 (local address 127.25.127.129:45879)
0619 08:23:36.392043 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:36.392046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:36.392138 (+ 92us) server_negotiation.cc:407] Connection header received
0619 08:23:36.392252 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:36.392253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:36.392264 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:36.392375 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:36.393055 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.393545 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.394297 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.394506 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.394581 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:36.396855 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:36.396856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:36.396857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:36.396864 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:36.398098 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:36.398448 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:36.398449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:36.398449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:36.398508 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:36.398697 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:36.398698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:36.398698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:36.398798 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:36.398903 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:36.399091 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 08:23:36.399171 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260619 08:23:36.400780 26873 tablet.cc:2369] T 144ae2ba886e46b3a981022b841dc2fd P 206c75ab6f0641c1be4208061af14acc: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 08:23:36 dist-test-slave-5rhg krb5kdc[26635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 08:23:36 dist-test-slave-5rhg krb5kdc[26635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.129@KRBTEST.COM
I20260619 08:23:36.421643 27156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:36.415003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38694 (local address 127.25.127.129:45879)
0619 08:23:36.415084 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:36.415086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:36.415098 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:23:36.415156 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:36.415157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:36.415167 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:36.415228 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:36.415766 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.416375 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.417066 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:36.417256 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:36.417340 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:36.419535 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:36.419537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:36.419537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:36.419543 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:36.420774 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:36.421071 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:36.421071 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:36.421072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:36.421118 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:36.421272 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:36.421273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:36.421273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:36.421356 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:36.421442 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:36.421524 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 08:23:36.421579 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260619 08:23:36.422128 26786 tablet_service.cc:2908] Rejecting scan request for tablet 144ae2ba886e46b3a981022b841dc2fd: Uninitialized: clean time has not yet been initialized
I20260619 08:23:36.423059 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 26743
I20260619 08:23:36.427747 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 26877
I20260619 08:23:36.432278 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 27011
I20260619 08:23:36.436651 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 26648
2026-06-19T08:23:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2028 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:23:36 dist-test-slave-5rhg krb5kdc[27181](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:23:36 dist-test-slave-5rhg krb5kdc[27181](info): set up 2 sockets
Jun 19 08:23:36 dist-test-slave-5rhg krb5kdc[27181](info): commencing operation
krb5kdc: starting...
W20260619 08:23:37.468509 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:23:37 dist-test-slave-5rhg krb5kdc[27181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:23:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:23:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:23:37.582227 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:41381
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:41381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:37.662856 27194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:37.663062 27194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:37.663085 27194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:37.664341 27194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:23:37.664391 27194 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 08:23:37.664407 27194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:37.664419 27194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:37.664433 27194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:23:37.664445 27194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:23:37.665992 27194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:41381
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:41381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:37.666245 27194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:37.666502 27194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:37.671361 27200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:37.671617 27194 server_base.cc:1034] running on GCE node
W20260619 08:23:37.671345 27199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:37.671391 27202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:37.672016 27194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:37.672396 27194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:37.673583 27194 hybrid_clock.cc:648] HybridClock initialized: now 1781857417673545 us; error 41 us; skew 500 ppm
Jun 19 08:23:37 dist-test-slave-5rhg krb5kdc[27181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857417, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:37.680735 27194 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:23:37.681141 27194 webserver.cc:460] Webserver started at http://127.25.127.190:42877/ using document root <none> and password file <none>
I20260619 08:23:37.681344 27194 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:37.681384 27194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:37.681483 27194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:37.682516 27194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "a49328ee990341c2aca22797dd19d93e"
format_stamp: "Formatted at 2026-06-19 08:23:37 on dist-test-slave-5rhg"
server_key: "e000adb927d1ab8ac5a167a02e61d6cf"
server_key_iv: "090277496a528d87f9208d36889057b8"
server_key_version: "encryptionkey@0"
I20260619 08:23:37.682821 27194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "a49328ee990341c2aca22797dd19d93e"
format_stamp: "Formatted at 2026-06-19 08:23:37 on dist-test-slave-5rhg"
server_key: "e000adb927d1ab8ac5a167a02e61d6cf"
server_key_iv: "090277496a528d87f9208d36889057b8"
server_key_version: "encryptionkey@0"
I20260619 08:23:37.684357 27194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:23:37.685159 27209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:37.685389 27194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:23:37.685500 27194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "a49328ee990341c2aca22797dd19d93e"
format_stamp: "Formatted at 2026-06-19 08:23:37 on dist-test-slave-5rhg"
server_key: "e000adb927d1ab8ac5a167a02e61d6cf"
server_key_iv: "090277496a528d87f9208d36889057b8"
server_key_version: "encryptionkey@0"
I20260619 08:23:37.685587 27194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:37.697319 27194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:37.699671 27194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:37.699833 27194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:37.704442 27194 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:41381
I20260619 08:23:37.704558 27271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:41381 every 8 connection(s)
I20260619 08:23:37.704855 27194 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:23:37.705540 27272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:37.706717 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 27194
I20260619 08:23:37.706828 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:23:37.706982 26110 external_mini_cluster.cc:1428] Setting key ca2a87930dfb81a0ef8b4d8a044bfce5
I20260619 08:23:37.708312 27272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e: Bootstrap starting.
I20260619 08:23:37.708971 27272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:37.709278 27272 log.cc:824] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:37.709993 27272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e: No bootstrap required, opened a new log
I20260619 08:23:37.711747 27272 raft_consensus.cc:348] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a49328ee990341c2aca22797dd19d93e" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41381 } }
I20260619 08:23:37.711922 27272 raft_consensus.cc:374] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:37.711970 27272 raft_consensus.cc:729] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a49328ee990341c2aca22797dd19d93e, State: Initialized, Role: FOLLOWER
I20260619 08:23:37.712095 27272 consensus_queue.cc:260] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a49328ee990341c2aca22797dd19d93e" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41381 } }
I20260619 08:23:37.712195 27272 raft_consensus.cc:388] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:23:37.712239 27272 raft_consensus.cc:482] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:23:37.712289 27272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:37.712818 27272 raft_consensus.cc:504] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a49328ee990341c2aca22797dd19d93e" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41381 } }
I20260619 08:23:37.712932 27272 leader_election.cc:302] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a49328ee990341c2aca22797dd19d93e; no voters:
I20260619 08:23:37.713102 27272 leader_election.cc:288] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:23:37.713146 27277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:37.713316 27277 raft_consensus.cc:686] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e [term 1 LEADER]: Becoming Leader. State: Replica: a49328ee990341c2aca22797dd19d93e, State: Running, Role: LEADER
I20260619 08:23:37.713363 27272 sys_catalog.cc:564] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:37.713425 27277 consensus_queue.cc:237] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a49328ee990341c2aca22797dd19d93e" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41381 } }
Jun 19 08:23:37 dist-test-slave-5rhg krb5kdc[27181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:37.713770 27277 sys_catalog.cc:455] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a49328ee990341c2aca22797dd19d93e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a49328ee990341c2aca22797dd19d93e" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41381 } } }
I20260619 08:23:37.713862 27277 sys_catalog.cc:458] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:37.713989 27277 sys_catalog.cc:455] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a49328ee990341c2aca22797dd19d93e. Latest consensus state: current_term: 1 leader_uuid: "a49328ee990341c2aca22797dd19d93e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a49328ee990341c2aca22797dd19d93e" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41381 } } }
I20260619 08:23:37.714051 27277 sys_catalog.cc:458] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:37.714359 27285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:23:37.714947 27285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:23:37.716933 27285 catalog_manager.cc:1347] Generated new cluster ID: d2218977731849a8bf7d58f56f9649ed
I20260619 08:23:37.716989 27285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:23:37.717978 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:37.707549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36688 (local address 127.25.127.190:41381)
0619 08:23:37.707751 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:37.707761 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:37.707798 (+ 37us) server_negotiation.cc:407] Connection header received
0619 08:23:37.708370 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:37.708379 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:37.708441 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:37.708592 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:37.709172 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:37.710165 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:37.710896 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:37.711332 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:37.711456 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:37.714265 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:37.714269 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:37.714269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:37.714280 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:37.716473 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:37.716876 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:37.716878 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:37.716879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:37.716972 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:37.717207 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:37.717208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:37.717208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:37.717429 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:37.717596 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:37.717668 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 08:23:37.717800 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260619 08:23:37.732998 27285 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:23:37.733734 27285 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:23:37.737957 27285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e: Generated new TSK 0
I20260619 08:23:37.738158 27285 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
I20260619 08:23:37.770833 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:41381
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:37.851939 27298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:37.852134 27298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:37.852217 27298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:37.853330 27298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:37.853364 27298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:37.853385 27298 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:23:37.853402 27298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:23:37.854729 27298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.127.190:41381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:37.854946 27298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:37.855154 27298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:37.859961 27306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:37.859958 27303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:37.859985 27304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:37.860277 27298 server_base.cc:1034] running on GCE node
I20260619 08:23:37.860468 27298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:37.860661 27298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:37.861797 27298 hybrid_clock.cc:648] HybridClock initialized: now 1781857417861784 us; error 28 us; skew 500 ppm
Jun 19 08:23:37 dist-test-slave-5rhg krb5kdc[27181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857417, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:37.864310 27298 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:23:37.864768 27298 webserver.cc:460] Webserver started at http://127.25.127.129:39843/ using document root <none> and password file <none>
I20260619 08:23:37.865007 27298 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:37.865073 27298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:37.865211 27298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:37.866214 27298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "2333a20009e84004b290128214d71e1c"
format_stamp: "Formatted at 2026-06-19 08:23:37 on dist-test-slave-5rhg"
server_key: "5cb4258c968992ee71c500725fd6d2b1"
server_key_iv: "7ac1633ffd0aa134e78d67de403361a8"
server_key_version: "encryptionkey@0"
I20260619 08:23:37.866570 27298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "2333a20009e84004b290128214d71e1c"
format_stamp: "Formatted at 2026-06-19 08:23:37 on dist-test-slave-5rhg"
server_key: "5cb4258c968992ee71c500725fd6d2b1"
server_key_iv: "7ac1633ffd0aa134e78d67de403361a8"
server_key_version: "encryptionkey@0"
I20260619 08:23:37.867827 27298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:23:37.868637 27313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:37.868816 27298 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:37.868898 27298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "2333a20009e84004b290128214d71e1c"
format_stamp: "Formatted at 2026-06-19 08:23:37 on dist-test-slave-5rhg"
server_key: "5cb4258c968992ee71c500725fd6d2b1"
server_key_iv: "7ac1633ffd0aa134e78d67de403361a8"
server_key_version: "encryptionkey@0"
I20260619 08:23:37.868988 27298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:37.887985 27298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:37.890123 27298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:37.890236 27298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:37.890695 27298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:37.890730 27298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:37.890758 27298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:37.890816 27298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:37.897997 27298 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:35459
I20260619 08:23:37.898080 27428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:35459 every 8 connection(s)
I20260619 08:23:37.898391 27298 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:23:37 dist-test-slave-5rhg krb5kdc[27181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857417, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:37.901369 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:37.891201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:33705 (local address 127.25.127.190:41381)
0619 08:23:37.891244 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:37.891246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:37.891518 (+ 272us) server_negotiation.cc:407] Connection header received
0619 08:23:37.892031 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:37.892032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:37.892044 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:37.892108 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:37.893112 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:37.893964 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:37.895802 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:37.896078 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:37.896134 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:37.899021 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:37.899022 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:37.899022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:37.899030 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:37.900248 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:37.900615 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:37.900616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:37.900616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:37.900667 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:37.900866 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:37.900867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:37.900867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:37.900971 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:37.901021 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:37.901238 (+ 217us) server_negotiation.cc:299] Negotiation successful
0619 08:23:37.901306 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:23:37.901546 27322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:37.891198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:41381 (local address 127.25.127.129:33705)
0619 08:23:37.891389 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:37.891436 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:37.891929 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:37.892256 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:37.892260 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:37.892665 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:37.892984 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:37.892992 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:37.894070 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:37.894071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:37.895708 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:37.895711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:37.896186 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:37.896188 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:37.896389 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:37.897077 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:37.897089 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:37.898934 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:37.900314 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:37.900320 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:37.900322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:37.900549 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:37.900722 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:37.900722 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:37.900722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:37.900815 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:37.901025 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:37.901029 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:37.901140 (+ 111us) client_negotiation.cc:769] Sending connection context
0619 08:23:37.901221 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 08:23:37.901335 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":117,"threads_started":1}
I20260619 08:23:37.902098 27429 heartbeater.cc:347] Connected to a master server at 127.25.127.190:41381
I20260619 08:23:37.902182 27429 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:37.902360 27429 heartbeater.cc:511] Master 127.25.127.190:41381 requested a full tablet report, sending...
I20260619 08:23:37.902891 27225 ts_manager.cc:194] Registered new tserver with Master: 2333a20009e84004b290128214d71e1c (127.25.127.129:35459)
I20260619 08:23:37.903718 27225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:33705
I20260619 08:23:37.905711 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 27298
I20260619 08:23:37.905826 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:23:37.905995 26110 external_mini_cluster.cc:1428] Setting key 769e0fa6bca3b8c45bef2a5875fcf89b
I20260619 08:23:37.908753 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:37.903088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:48381 (local address 127.25.127.190:41381)
0619 08:23:37.903124 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:37.903126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:37.903236 (+ 110us) server_negotiation.cc:407] Connection header received
0619 08:23:37.903259 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:37.903260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:37.903269 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:37.903336 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:37.903610 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:37.904268 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:37.904927 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:37.905104 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:37.905158 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:37.906098 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:37.906100 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:37.906100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:37.906107 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:37.907699 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:37.908014 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:37.908015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:37.908016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:37.908078 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:37.908326 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:37.908327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:37.908327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:37.908428 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:37.908480 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:37.908645 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 08:23:37.908699 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 08:23:37.908856 27435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:37.903040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:41381 (local address 127.25.127.129:48381)
0619 08:23:37.903174 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:37.903182 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:37.903227 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:37.903337 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:37.903338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:37.903506 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:37.903553 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:37.903554 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:37.904343 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:37.904344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:37.904855 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:37.904856 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:37.905170 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:37.905171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:37.905222 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:37.905652 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:37.905657 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:37.905989 (+ 332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:37.907779 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:37.907780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:37.907780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:37.907943 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:37.908174 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:37.908175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:37.908175 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:37.908257 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:37.908494 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:37.908494 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:37.908546 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 08:23:37.908628 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 08:23:37.908731 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":6,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.25.127.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
I20260619 08:23:37.928572 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:41381
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:38.010046 27438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:38.010238 27438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:38.010262 27438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:38.011446 27438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:38.011488 27438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:38.011508 27438 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:23:38.011567 27438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:23:38.013008 27438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.127.190:41381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:38.013254 27438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:38.013496 27438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:38.018478 27444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:38.018512 27443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:38.018541 27438 server_base.cc:1034] running on GCE node
W20260619 08:23:38.018489 27446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:38.018879 27438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:38.019170 27438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:38.020313 27438 hybrid_clock.cc:648] HybridClock initialized: now 1781857418020293 us; error 42 us; skew 500 ppm
Jun 19 08:23:38 dist-test-slave-5rhg krb5kdc[27181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857418, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:38.022768 27438 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:23:38.023188 27438 webserver.cc:460] Webserver started at http://127.25.127.130:38953/ using document root <none> and password file <none>
I20260619 08:23:38.023420 27438 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:38.023463 27438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:38.023603 27438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:38.024694 27438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "9a8d2e33a25e41e29d4937901d07e2e6"
format_stamp: "Formatted at 2026-06-19 08:23:38 on dist-test-slave-5rhg"
server_key: "58c660751bc6aff1f205f91d17781488"
server_key_iv: "1f33e50b26a4dae52fe71af9818b3e1c"
server_key_version: "encryptionkey@0"
I20260619 08:23:38.025061 27438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "9a8d2e33a25e41e29d4937901d07e2e6"
format_stamp: "Formatted at 2026-06-19 08:23:38 on dist-test-slave-5rhg"
server_key: "58c660751bc6aff1f205f91d17781488"
server_key_iv: "1f33e50b26a4dae52fe71af9818b3e1c"
server_key_version: "encryptionkey@0"
I20260619 08:23:38.026327 27438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:23:38.026996 27453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:38.027163 27438 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:38.027246 27438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "9a8d2e33a25e41e29d4937901d07e2e6"
format_stamp: "Formatted at 2026-06-19 08:23:38 on dist-test-slave-5rhg"
server_key: "58c660751bc6aff1f205f91d17781488"
server_key_iv: "1f33e50b26a4dae52fe71af9818b3e1c"
server_key_version: "encryptionkey@0"
I20260619 08:23:38.027318 27438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:38.054072 27438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:38.056888 27438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:38.057045 27438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:38.057560 27438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:38.057626 27438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:38.057662 27438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:38.057680 27438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 08:23:38 dist-test-slave-5rhg krb5kdc[27181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857418, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:38.065240 27438 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:40697
I20260619 08:23:38.065330 27568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:40697 every 8 connection(s)
I20260619 08:23:38.065645 27438 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:23:38.068032 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.058469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:43905 (local address 127.25.127.190:41381)
0619 08:23:38.058518 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.058520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.058795 (+ 275us) server_negotiation.cc:407] Connection header received
0619 08:23:38.059439 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.059441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.059456 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.059490 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:38.060499 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.061345 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.062380 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.062737 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.062820 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.065026 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:38.065028 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:38.065028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:38.065036 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:38.066643 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:38.067064 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:38.067065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:38.067066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:38.067130 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:38.067376 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:38.067377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:38.067377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:38.067495 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:38.067573 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.067861 (+ 288us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.067937 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:23:38.068243 27462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.058404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:41381 (local address 127.25.127.130:43905)
0619 08:23:38.058661 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:38.058702 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:38.059305 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:38.059680 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:38.059685 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:38.060062 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:38.060385 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.060392 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.061493 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.061494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.062302 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.062304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.062872 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.062873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.063003 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.063516 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:38.063524 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:38.064932 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:38.066753 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:38.066759 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:38.066761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:38.066989 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:38.067197 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:38.067198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:38.067199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:38.067310 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:38.067565 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:38.067571 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:38.067706 (+ 135us) client_negotiation.cc:769] Sending connection context
0619 08:23:38.067836 (+ 130us) client_negotiation.cc:240] Negotiation successful
0619 08:23:38.068000 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
I20260619 08:23:38.068771 27569 heartbeater.cc:347] Connected to a master server at 127.25.127.190:41381
I20260619 08:23:38.068843 27569 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:38.069024 27569 heartbeater.cc:511] Master 127.25.127.190:41381 requested a full tablet report, sending...
I20260619 08:23:38.069396 27226 ts_manager.cc:194] Registered new tserver with Master: 9a8d2e33a25e41e29d4937901d07e2e6 (127.25.127.130:40697)
I20260619 08:23:38.069783 27226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:43905
I20260619 08:23:38.073035 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 27438
I20260619 08:23:38.073143 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:23:38.073330 26110 external_mini_cluster.cc:1428] Setting key 72ec4a5f31ec85dbd82fd3373d523ea2
I20260619 08:23:38.075616 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.069887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:52287 (local address 127.25.127.190:41381)
0619 08:23:38.069916 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.069918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.070030 (+ 112us) server_negotiation.cc:407] Connection header received
0619 08:23:38.070123 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.070124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.070133 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.070180 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:38.070517 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.070955 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.071665 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.071837 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.071893 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.072731 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:38.072732 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:38.072732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:38.072738 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:38.074495 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:38.074807 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:38.074808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:38.074808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:38.074866 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:38.075040 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:38.075041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:38.075042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:38.075238 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:38.075315 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.075492 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.075562 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260619 08:23:38.075718 27575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.069808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:41381 (local address 127.25.127.130:52287)
0619 08:23:38.069981 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:38.069992 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:38.070044 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:38.070187 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:38.070188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:38.070373 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:38.070447 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.070449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.071046 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.071047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.071603 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.071605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.071895 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.071896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.071949 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.072354 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:38.072358 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:38.072654 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:38.074579 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:38.074580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:38.074580 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:38.074741 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:38.074931 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:38.074931 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:38.074931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:38.074981 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:38.075320 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:38.075321 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:38.075376 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 08:23:38.075453 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 08:23:38.075576 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for kudu/127.25.127.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
I20260619 08:23:38.096573 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:41381
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:36191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:38.177210 27578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:38.177409 27578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:38.177433 27578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:38.178575 27578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:38.178617 27578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:38.178637 27578 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:23:38.178655 27578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:23:38.180100 27578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:36191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.127.190:41381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:38.180424 27578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:38.180653 27578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:38.185478 27583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:38.185554 27584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:38.185575 27578 server_base.cc:1034] running on GCE node
W20260619 08:23:38.185478 27586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:38.185918 27578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:38.186178 27578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:38.187340 27578 hybrid_clock.cc:648] HybridClock initialized: now 1781857418187337 us; error 36 us; skew 500 ppm
Jun 19 08:23:38 dist-test-slave-5rhg krb5kdc[27181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857418, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:38.189926 27578 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:23:38.190300 27578 webserver.cc:460] Webserver started at http://127.25.127.131:37481/ using document root <none> and password file <none>
I20260619 08:23:38.190546 27578 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:38.190618 27578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:38.190766 27578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:38.191788 27578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "293161cad44d4efcb40224988580abc8"
format_stamp: "Formatted at 2026-06-19 08:23:38 on dist-test-slave-5rhg"
server_key: "c50347d031567c273dc0cca87864d173"
server_key_iv: "d828955666e762dc5f928234ee85cc5f"
server_key_version: "encryptionkey@0"
I20260619 08:23:38.192179 27578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "293161cad44d4efcb40224988580abc8"
format_stamp: "Formatted at 2026-06-19 08:23:38 on dist-test-slave-5rhg"
server_key: "c50347d031567c273dc0cca87864d173"
server_key_iv: "d828955666e762dc5f928234ee85cc5f"
server_key_version: "encryptionkey@0"
I20260619 08:23:38.193446 27578 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:38.194139 27593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:38.194308 27578 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:38.194402 27578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "293161cad44d4efcb40224988580abc8"
format_stamp: "Formatted at 2026-06-19 08:23:38 on dist-test-slave-5rhg"
server_key: "c50347d031567c273dc0cca87864d173"
server_key_iv: "d828955666e762dc5f928234ee85cc5f"
server_key_version: "encryptionkey@0"
I20260619 08:23:38.194490 27578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:38.205590 27578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:38.207427 27578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:38.207582 27578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:38.208060 27578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:38.208122 27578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:38.208202 27578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:38.208228 27578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 08:23:38 dist-test-slave-5rhg krb5kdc[27181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857418, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:38.216055 27578 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:35615
I20260619 08:23:38.216208 27708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:35615 every 8 connection(s)
I20260619 08:23:38.216538 27578 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:23:38.218458 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.208633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:41507 (local address 127.25.127.190:41381)
0619 08:23:38.208705 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.208707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.208989 (+ 282us) server_negotiation.cc:407] Connection header received
0619 08:23:38.209671 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.209672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.209682 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.209730 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:38.210698 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.211220 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.212270 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.212469 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.212555 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.215428 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:38.215429 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:38.215430 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:38.215438 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:38.217113 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:38.217504 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:38.217505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:38.217506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:38.217558 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:38.217826 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:38.217827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:38.217827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:38.217947 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:38.218017 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.218286 (+ 269us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.218363 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:23:38.218653 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.208603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:41381 (local address 127.25.127.131:41507)
0619 08:23:38.208807 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:38.208855 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:38.209556 (+ 701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:38.209810 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:38.209815 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:38.210237 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:38.210544 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.210552 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.211330 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.211331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.212198 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.212201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.212566 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.212567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.212731 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.213301 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:38.213313 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:38.215333 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:38.217203 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:38.217209 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:38.217210 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:38.217429 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:38.217638 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:38.217638 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:38.217639 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:38.217742 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:38.218020 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:38.218027 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:38.218160 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 08:23:38.218266 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 08:23:38.218431 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":103,"threads_started":1}
I20260619 08:23:38.219132 27709 heartbeater.cc:347] Connected to a master server at 127.25.127.190:41381
I20260619 08:23:38.219215 27709 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:38.219396 27709 heartbeater.cc:511] Master 127.25.127.190:41381 requested a full tablet report, sending...
I20260619 08:23:38.219800 27225 ts_manager.cc:194] Registered new tserver with Master: 293161cad44d4efcb40224988580abc8 (127.25.127.131:35615)
I20260619 08:23:38.220356 27225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:41507
I20260619 08:23:38.220814 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 27578
I20260619 08:23:38.220912 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:23:38.221081 26110 external_mini_cluster.cc:1428] Setting key ef296dfa1b7c560d17eae682524efb59
I20260619 08:23:38.221834 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:23:38.226241 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.220298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:38605 (local address 127.25.127.190:41381)
0619 08:23:38.220337 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.220339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.220353 (+ 14us) server_negotiation.cc:407] Connection header received
0619 08:23:38.220429 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.220429 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.220438 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.220485 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:38.220946 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.221606 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.222398 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.222628 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.222713 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.223660 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:38.223662 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:38.223662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:38.223670 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:38.225115 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:38.225485 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:38.225486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:38.225487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:38.225538 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:38.225742 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:38.225743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:38.225743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:38.225852 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:38.225926 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.226083 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.226153 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:23:38.226261 27715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.220186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:41381 (local address 127.25.127.131:38605)
0619 08:23:38.220303 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:38.220316 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:38.220366 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:38.220530 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:38.220531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:38.220769 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:38.220851 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.220854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.221732 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.221732 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.222305 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.222307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.222717 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.222718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.222792 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.223235 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:38.223239 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:38.223572 (+ 333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:38.225220 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:38.225221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:38.225221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:38.225399 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:38.225612 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:38.225613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:38.225613 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:38.225669 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:38.225919 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:38.225920 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:38.225965 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 08:23:38.226054 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 08:23:38.226155 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":1,"thread_start_us":65,"threads_started":1}
Jun 19 08:23:38 dist-test-slave-5rhg krb5kdc[27181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857418, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 08:23:38 dist-test-slave-5rhg krb5kdc[27181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857418, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:38.244040 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.237383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.25.127.190:41381)
0619 08:23:38.237422 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.237424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.237433 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:23:38.237477 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.237478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.237486 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.237537 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:38.238236 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.238725 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.239443 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.239686 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.239742 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.241814 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:38.241815 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:38.241815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:38.241821 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:38.243172 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:38.243476 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:38.243476 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:38.243477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:38.243527 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:38.243676 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:38.243676 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:38.243677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:38.243767 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:38.243836 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.243921 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.243976 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:23:38.245610 27225 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:23:38.247148 27225 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:23:38.253798 27727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.249587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:40697 (local address 127.0.0.1:50006)
0619 08:23:38.249762 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:38.249788 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:38.249854 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:38.250388 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:38.250389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:38.250392 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:38.250450 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.250451 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.251400 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.251401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.252559 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.252561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.253490 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.253492 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.253572 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.253586 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 08:23:38.253660 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 08:23:38.253717 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":73,"threads_started":1}
I20260619 08:23:38.253839 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.249998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50006 (local address 127.25.127.130:40697)
0619 08:23:38.250163 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.250167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.250175 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:23:38.250207 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.250208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.250241 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.250321 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:38.250512 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.251310 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.252644 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.253385 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.253474 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.253529 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.253687 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.253728 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":67,"threads_started":1}
I20260619 08:23:38.254038 27728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.249591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:35459 (local address 127.0.0.1:48250)
0619 08:23:38.249955 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:38.249963 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:38.249994 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:38.250635 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:38.250636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:38.250637 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:38.250671 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.250672 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.251858 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.251859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.252929 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.252931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.253811 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.253812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.253876 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.253881 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:23:38.253913 (+ 32us) client_negotiation.cc:240] Negotiation successful
0619 08:23:38.253971 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":87,"threads_started":1}
I20260619 08:23:38.253963 27726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.249578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:35615 (local address 127.0.0.1:32804)
0619 08:23:38.249746 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:38.249789 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:38.249854 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:38.250351 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:38.250354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:38.250356 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:38.250428 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.250430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.251799 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.251800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.252832 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.252835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.253638 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.253639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.253696 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.253699 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:23:38.253743 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 08:23:38.253776 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":22,"thread_start_us":56,"threads_started":1}
I20260619 08:23:38.254254 27729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.249780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48250 (local address 127.25.127.129:35459)
0619 08:23:38.250496 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.250500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.250509 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:23:38.250544 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.250545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.250561 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.250642 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:38.250724 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.251775 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.252995 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.253712 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.254019 (+ 307us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.254084 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.254110 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.254150 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":83,"threads_started":1}
I20260619 08:23:38.254611 27730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.249826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32804 (local address 127.25.127.131:35615)
0619 08:23:38.250146 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.250150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.250181 (+ 31us) server_negotiation.cc:407] Connection header received
0619 08:23:38.250223 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.250225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.250240 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.250345 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:38.250599 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.251704 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.252922 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.253478 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.253552 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.253605 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.253838 (+ 233us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.254514 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":68,"threads_started":1}
I20260619 08:23:38.254652 27503 tablet_service.cc:1467] Processing CreateTablet for tablet ddae829c2226466285f0682108c3c502 (DEFAULT_TABLE table=test-table [id=0c99df475a1e41bd826dae9a17573781]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:38.254894 27349 tablet_service.cc:1467] Processing CreateTablet for tablet ddae829c2226466285f0682108c3c502 (DEFAULT_TABLE table=test-table [id=0c99df475a1e41bd826dae9a17573781]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:38.255055 27503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddae829c2226466285f0682108c3c502. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:38.255155 27349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddae829c2226466285f0682108c3c502. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:38.255205 27643 tablet_service.cc:1467] Processing CreateTablet for tablet ddae829c2226466285f0682108c3c502 (DEFAULT_TABLE table=test-table [id=0c99df475a1e41bd826dae9a17573781]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:38.255484 27643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddae829c2226466285f0682108c3c502. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:38.258102 27732 tablet_bootstrap.cc:492] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c: Bootstrap starting.
I20260619 08:23:38.258760 27734 tablet_bootstrap.cc:492] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6: Bootstrap starting.
I20260619 08:23:38.258980 27732 tablet_bootstrap.cc:654] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:38.259380 27732 log.cc:824] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:38.259456 27734 tablet_bootstrap.cc:654] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:38.259797 27733 tablet_bootstrap.cc:492] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8: Bootstrap starting.
I20260619 08:23:38.259840 27734 log.cc:824] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:38.260203 27732 tablet_bootstrap.cc:492] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c: No bootstrap required, opened a new log
I20260619 08:23:38.260285 27732 ts_tablet_manager.cc:1397] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:23:38.260771 27734 tablet_bootstrap.cc:492] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6: No bootstrap required, opened a new log
I20260619 08:23:38.260864 27733 tablet_bootstrap.cc:654] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:38.260854 27734 ts_tablet_manager.cc:1397] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260619 08:23:38.261190 27733 log.cc:824] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:38.261925 27733 tablet_bootstrap.cc:492] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8: No bootstrap required, opened a new log
I20260619 08:23:38.261994 27733 ts_tablet_manager.cc:1397] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 08:23:38.262120 27732 raft_consensus.cc:348] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } }
I20260619 08:23:38.262274 27732 raft_consensus.cc:374] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:38.262305 27732 raft_consensus.cc:729] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2333a20009e84004b290128214d71e1c, State: Initialized, Role: FOLLOWER
I20260619 08:23:38.262428 27732 consensus_queue.cc:260] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } }
I20260619 08:23:38.262687 27732 ts_tablet_manager.cc:1428] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:23:38.262733 27429 heartbeater.cc:503] Master 127.25.127.190:41381 was elected leader, sending a full tablet report...
I20260619 08:23:38.263150 27734 raft_consensus.cc:348] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } }
I20260619 08:23:38.263263 27734 raft_consensus.cc:374] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:38.263283 27734 raft_consensus.cc:729] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a8d2e33a25e41e29d4937901d07e2e6, State: Initialized, Role: FOLLOWER
I20260619 08:23:38.263388 27734 consensus_queue.cc:260] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } }
I20260619 08:23:38.263648 27734 ts_tablet_manager.cc:1428] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 08:23:38.263790 27569 heartbeater.cc:503] Master 127.25.127.190:41381 was elected leader, sending a full tablet report...
I20260619 08:23:38.264011 27733 raft_consensus.cc:348] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } }
I20260619 08:23:38.264181 27733 raft_consensus.cc:374] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:38.264215 27733 raft_consensus.cc:729] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 293161cad44d4efcb40224988580abc8, State: Initialized, Role: FOLLOWER
I20260619 08:23:38.264320 27733 consensus_queue.cc:260] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } }
I20260619 08:23:38.264585 27733 ts_tablet_manager.cc:1428] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 08:23:38.264679 27709 heartbeater.cc:503] Master 127.25.127.190:41381 was elected leader, sending a full tablet report...
W20260619 08:23:38.316248 27570 tablet.cc:2369] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:23:38.393908 27739 raft_consensus.cc:482] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:23:38.394027 27739 raft_consensus.cc:504] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } }
I20260619 08:23:38.394456 27739 leader_election.cc:288] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 293161cad44d4efcb40224988580abc8 (127.25.127.131:35615), 2333a20009e84004b290128214d71e1c (127.25.127.129:35459)
I20260619 08:23:38.398442 27730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.394838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:42077 (local address 127.25.127.131:35615)
0619 08:23:38.394917 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.394919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.395232 (+ 313us) server_negotiation.cc:407] Connection header received
0619 08:23:38.395301 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.395302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.395312 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.395385 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:38.395670 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.396309 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.397507 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.398065 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.398131 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.398141 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.398337 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.398374 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 08:23:38.398605 27462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.394679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:35459 (local address 127.25.127.130:47129)
0619 08:23:38.394749 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:38.394760 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:38.394811 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:38.395107 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:38.395109 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:38.395110 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:38.395189 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.395191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.396037 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.396038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.397532 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.397535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.398271 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.398271 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.398389 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.398394 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:23:38.398447 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 08:23:38.398495 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 08:23:38.398504 27741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.394685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:35615 (local address 127.25.127.130:42077)
0619 08:23:38.395145 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:38.395157 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:38.395225 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:38.395402 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:38.395403 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:38.395405 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:38.395588 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.395591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.396404 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.396405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.397412 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.397414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.398189 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.398190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.398276 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.398281 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:23:38.398331 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 08:23:38.398410 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":213,"thread_start_us":156,"threads_started":1}
I20260619 08:23:38.398974 27729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.394928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:47129 (local address 127.25.127.129:35459)
0619 08:23:38.394991 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.394993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.394999 (+ 6us) server_negotiation.cc:407] Connection header received
0619 08:23:38.395024 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.395025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.395032 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.395103 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:38.395288 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.395951 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.397650 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.398195 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.398260 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.398269 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.398729 (+ 460us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.398874 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:23:38.399044 27663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddae829c2226466285f0682108c3c502" candidate_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "293161cad44d4efcb40224988580abc8" is_pre_election: true
W20260619 08:23:38.399173 27430 tablet.cc:2369] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:23:38.399245 27663 raft_consensus.cc:2454] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a8d2e33a25e41e29d4937901d07e2e6 in term 0.
I20260619 08:23:38.399468 27456 leader_election.cc:302] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 293161cad44d4efcb40224988580abc8, 9a8d2e33a25e41e29d4937901d07e2e6; no voters:
I20260619 08:23:38.399611 27739 raft_consensus.cc:2781] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:23:38.399694 27739 raft_consensus.cc:482] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:23:38.399726 27739 raft_consensus.cc:3037] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:38.399696 27383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddae829c2226466285f0682108c3c502" candidate_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2333a20009e84004b290128214d71e1c" is_pre_election: true
I20260619 08:23:38.399852 27383 raft_consensus.cc:2454] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a8d2e33a25e41e29d4937901d07e2e6 in term 0.
I20260619 08:23:38.400825 27739 raft_consensus.cc:504] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } }
I20260619 08:23:38.401093 27739 leader_election.cc:288] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [CANDIDATE]: Term 1 election: Requested vote from peers 293161cad44d4efcb40224988580abc8 (127.25.127.131:35615), 2333a20009e84004b290128214d71e1c (127.25.127.129:35459)
I20260619 08:23:38.401113 27663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddae829c2226466285f0682108c3c502" candidate_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "293161cad44d4efcb40224988580abc8"
I20260619 08:23:38.401129 27383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddae829c2226466285f0682108c3c502" candidate_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2333a20009e84004b290128214d71e1c"
I20260619 08:23:38.401192 27663 raft_consensus.cc:3037] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:38.401224 27383 raft_consensus.cc:3037] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:38.402145 27663 raft_consensus.cc:2454] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8d2e33a25e41e29d4937901d07e2e6 in term 1.
I20260619 08:23:38.402328 27456 leader_election.cc:302] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 293161cad44d4efcb40224988580abc8, 9a8d2e33a25e41e29d4937901d07e2e6; no voters:
I20260619 08:23:38.402413 27739 raft_consensus.cc:2781] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:38.402452 27383 raft_consensus.cc:2454] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8d2e33a25e41e29d4937901d07e2e6 in term 1.
I20260619 08:23:38.402644 27739 raft_consensus.cc:686] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 1 LEADER]: Becoming Leader. State: Replica: 9a8d2e33a25e41e29d4937901d07e2e6, State: Running, Role: LEADER
I20260619 08:23:38.402748 27739 consensus_queue.cc:237] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } }
I20260619 08:23:38.403703 27225 catalog_manager.cc:5495] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a8d2e33a25e41e29d4937901d07e2e6 (127.25.127.130). New cstate: current_term: 1 leader_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } health_report { overall_health: HEALTHY } } }
I20260619 08:23:38.439431 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.436636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50010 (local address 127.25.127.130:40697)
0619 08:23:38.436699 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.436701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.436711 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:23:38.436801 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.436802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.436821 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.436886 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:23:38.437069 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.437627 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.438453 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.438657 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.438735 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.438976 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:23:38.439078 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:23:38.439176 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.439222 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.439242 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 08:23:38.441397 27383 raft_consensus.cc:1261] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c [term 1 FOLLOWER]: Refusing update from remote peer 9a8d2e33a25e41e29d4937901d07e2e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:38.441747 27663 raft_consensus.cc:1261] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8 [term 1 FOLLOWER]: Refusing update from remote peer 9a8d2e33a25e41e29d4937901d07e2e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:38.441803 27742 consensus_queue.cc:1035] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:38.442015 27742 consensus_queue.cc:1035] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:38.446094 27748 mvcc.cc:205] Tried to move back new op lower bound from 7298487985932922880 to 7298487985778335744. Current Snapshot: MvccSnapshot[applied={T|T < 7298487985932922880}]
I20260619 08:23:38.447605 27225 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:23:38.447707 27225 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:23:38.448540 27225 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e: Sending DeleteTablet for 3 replicas of tablet ddae829c2226466285f0682108c3c502
I20260619 08:23:38.448840 27643 tablet_service.cc:1514] Processing DeleteTablet for tablet ddae829c2226466285f0682108c3c502 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:38 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:32804
I20260619 08:23:38.448907 27349 tablet_service.cc:1514] Processing DeleteTablet for tablet ddae829c2226466285f0682108c3c502 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:38 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:48250
I20260619 08:23:38.449230 27503 tablet_service.cc:1514] Processing DeleteTablet for tablet ddae829c2226466285f0682108c3c502 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:38 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:50006
I20260619 08:23:38.449283 27755 tablet_replica.cc:330] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c: stopping tablet replica
I20260619 08:23:38.449307 27225 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 08:23:38.449384 27755 raft_consensus.cc:2229] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260619 08:23:38.449430 27225 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:23:38.449483 27755 raft_consensus.cc:2258] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:38.449640 27754 tablet_replica.cc:330] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8: stopping tablet replica
I20260619 08:23:38.449782 27754 raft_consensus.cc:2229] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:23:38.449816 27756 tablet_replica.cc:330] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6: stopping tablet replica
I20260619 08:23:38.449927 27756 raft_consensus.cc:2229] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:23:38.449955 27754 raft_consensus.cc:2258] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:38.450073 27756 raft_consensus.cc:2258] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:38.450639 27754 ts_tablet_manager.cc:1905] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:38.450752 27756 ts_tablet_manager.cc:1905] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:38.450913 27349 tablet_service.cc:1467] Processing CreateTablet for tablet c38923a8201142ff8a6e2cc75116d393 (DEFAULT_TABLE table=test-table [id=213ce8cc5ed241ce99e5d73686342da3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:38.451059 27349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c38923a8201142ff8a6e2cc75116d393. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:38.452527 27643 tablet_service.cc:1467] Processing CreateTablet for tablet c38923a8201142ff8a6e2cc75116d393 (DEFAULT_TABLE table=test-table [id=213ce8cc5ed241ce99e5d73686342da3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:38.452694 27643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c38923a8201142ff8a6e2cc75116d393. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:38.452701 27754 ts_tablet_manager.cc:1918] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:23:38.452793 27754 log.cc:1196] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/ddae829c2226466285f0682108c3c502
I20260619 08:23:38.452971 27756 ts_tablet_manager.cc:1918] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:23:38.453032 27756 log.cc:1196] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/ddae829c2226466285f0682108c3c502
I20260619 08:23:38.453042 27754 ts_tablet_manager.cc:1939] T ddae829c2226466285f0682108c3c502 P 293161cad44d4efcb40224988580abc8: Deleting consensus metadata
I20260619 08:23:38.453254 27756 ts_tablet_manager.cc:1939] T ddae829c2226466285f0682108c3c502 P 9a8d2e33a25e41e29d4937901d07e2e6: Deleting consensus metadata
I20260619 08:23:38.453939 27733 tablet_bootstrap.cc:492] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8: Bootstrap starting.
I20260619 08:23:38.453959 27732 tablet_bootstrap.cc:492] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c: Bootstrap starting.
I20260619 08:23:38.454324 27732 tablet_bootstrap.cc:654] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:38.454384 27733 tablet_bootstrap.cc:654] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:38.454744 27212 catalog_manager.cc:4841] TS 293161cad44d4efcb40224988580abc8 (127.25.127.131:35615): tablet ddae829c2226466285f0682108c3c502 (table test-table [id=0c99df475a1e41bd826dae9a17573781]) successfully deleted
I20260619 08:23:38.454944 27732 tablet_bootstrap.cc:492] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c: No bootstrap required, opened a new log
I20260619 08:23:38.454984 27732 ts_tablet_manager.cc:1397] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:38.454952 27211 catalog_manager.cc:4841] TS 9a8d2e33a25e41e29d4937901d07e2e6 (127.25.127.130:40697): tablet ddae829c2226466285f0682108c3c502 (table test-table [id=0c99df475a1e41bd826dae9a17573781]) successfully deleted
I20260619 08:23:38.455231 27732 raft_consensus.cc:348] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } }
I20260619 08:23:38.455292 27732 raft_consensus.cc:374] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:38.455305 27732 raft_consensus.cc:729] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2333a20009e84004b290128214d71e1c, State: Initialized, Role: FOLLOWER
I20260619 08:23:38.455302 27503 tablet_service.cc:1467] Processing CreateTablet for tablet c38923a8201142ff8a6e2cc75116d393 (DEFAULT_TABLE table=test-table [id=213ce8cc5ed241ce99e5d73686342da3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:38.455358 27732 consensus_queue.cc:260] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } }
I20260619 08:23:38.455408 27503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c38923a8201142ff8a6e2cc75116d393. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:38.455453 27732 ts_tablet_manager.cc:1428] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:38.455714 27733 tablet_bootstrap.cc:492] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8: No bootstrap required, opened a new log
I20260619 08:23:38.455753 27733 ts_tablet_manager.cc:1397] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260619 08:23:38.455894 27733 raft_consensus.cc:348] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } }
I20260619 08:23:38.455956 27733 raft_consensus.cc:374] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:38.455971 27733 raft_consensus.cc:729] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 293161cad44d4efcb40224988580abc8, State: Initialized, Role: FOLLOWER
I20260619 08:23:38.456025 27733 consensus_queue.cc:260] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } }
I20260619 08:23:38.456102 27733 ts_tablet_manager.cc:1428] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:38.456780 27734 tablet_bootstrap.cc:492] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6: Bootstrap starting.
I20260619 08:23:38.457175 27734 tablet_bootstrap.cc:654] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:38.457934 27734 tablet_bootstrap.cc:492] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6: No bootstrap required, opened a new log
I20260619 08:23:38.457973 27734 ts_tablet_manager.cc:1397] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 08:23:38.458078 27734 raft_consensus.cc:348] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } }
I20260619 08:23:38.458125 27734 raft_consensus.cc:374] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:38.458137 27734 raft_consensus.cc:729] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a8d2e33a25e41e29d4937901d07e2e6, State: Initialized, Role: FOLLOWER
I20260619 08:23:38.458192 27734 consensus_queue.cc:260] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } }
I20260619 08:23:38.458283 27734 ts_tablet_manager.cc:1428] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:38.458647 27755 ts_tablet_manager.cc:1905] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:38.460381 27755 ts_tablet_manager.cc:1918] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:23:38.460450 27755 log.cc:1196] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/wal/wals/ddae829c2226466285f0682108c3c502
I20260619 08:23:38.460696 27755 ts_tablet_manager.cc:1939] T ddae829c2226466285f0682108c3c502 P 2333a20009e84004b290128214d71e1c: Deleting consensus metadata
I20260619 08:23:38.461072 27213 catalog_manager.cc:4841] TS 2333a20009e84004b290128214d71e1c (127.25.127.129:35459): tablet ddae829c2226466285f0682108c3c502 (table test-table [id=0c99df475a1e41bd826dae9a17573781]) successfully deleted
W20260619 08:23:38.467166 27710 tablet.cc:2369] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:23:38.696954 27740 raft_consensus.cc:482] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:23:38.697079 27740 raft_consensus.cc:504] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } }
I20260619 08:23:38.697484 27740 leader_election.cc:288] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2333a20009e84004b290128214d71e1c (127.25.127.129:35459), 9a8d2e33a25e41e29d4937901d07e2e6 (127.25.127.130:40697)
I20260619 08:23:38.700455 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.697663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:40697 (local address 127.25.127.131:51663)
0619 08:23:38.697752 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:38.697765 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:38.697809 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:38.697957 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:38.697958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:38.697960 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:38.698038 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.698039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.698745 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.698746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.699580 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.699582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.700211 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.700212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.700296 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.700300 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:23:38.700355 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 08:23:38.700400 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260619 08:23:38.700498 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.697725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:51663 (local address 127.25.127.130:40697)
0619 08:23:38.697789 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.697792 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.697805 (+ 13us) server_negotiation.cc:407] Connection header received
0619 08:23:38.697874 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.697875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.697889 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.697943 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:38.698120 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.698668 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.699671 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.700104 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.700250 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.700259 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.700366 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.700406 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:23:38.700613 27729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.697757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:51079 (local address 127.25.127.129:35459)
0619 08:23:38.697804 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.697806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.697923 (+ 117us) server_negotiation.cc:407] Connection header received
0619 08:23:38.697995 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.697996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.698005 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.698040 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:38.698317 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.698829 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.699830 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.700261 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.700334 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.700343 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.700509 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.700553 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:23:38.700819 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.697669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:35459 (local address 127.25.127.131:51079)
0619 08:23:38.697855 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:38.697866 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:38.697934 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:38.698122 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:38.698123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:38.698124 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:38.698202 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.698204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.698919 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.698920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.699747 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.699749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.700358 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.700360 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.700428 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.700432 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:23:38.700490 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 08:23:38.700734 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":41,"thread_start_us":65,"threads_started":1}
I20260619 08:23:38.700928 27523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c38923a8201142ff8a6e2cc75116d393" candidate_uuid: "293161cad44d4efcb40224988580abc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" is_pre_election: true
I20260619 08:23:38.701042 27383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c38923a8201142ff8a6e2cc75116d393" candidate_uuid: "293161cad44d4efcb40224988580abc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2333a20009e84004b290128214d71e1c" is_pre_election: true
I20260619 08:23:38.701102 27523 raft_consensus.cc:2454] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 293161cad44d4efcb40224988580abc8 in term 0.
I20260619 08:23:38.701119 27383 raft_consensus.cc:2454] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 293161cad44d4efcb40224988580abc8 in term 0.
I20260619 08:23:38.701289 27595 leader_election.cc:302] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 293161cad44d4efcb40224988580abc8, 9a8d2e33a25e41e29d4937901d07e2e6; no voters:
I20260619 08:23:38.701436 27740 raft_consensus.cc:2781] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:23:38.701490 27740 raft_consensus.cc:482] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:23:38.701510 27740 raft_consensus.cc:3037] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:38.702296 27740 raft_consensus.cc:504] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } }
I20260619 08:23:38.702452 27740 leader_election.cc:288] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [CANDIDATE]: Term 1 election: Requested vote from peers 2333a20009e84004b290128214d71e1c (127.25.127.129:35459), 9a8d2e33a25e41e29d4937901d07e2e6 (127.25.127.130:40697)
I20260619 08:23:38.702565 27523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c38923a8201142ff8a6e2cc75116d393" candidate_uuid: "293161cad44d4efcb40224988580abc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8d2e33a25e41e29d4937901d07e2e6"
I20260619 08:23:38.702565 27383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c38923a8201142ff8a6e2cc75116d393" candidate_uuid: "293161cad44d4efcb40224988580abc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2333a20009e84004b290128214d71e1c"
I20260619 08:23:38.702634 27523 raft_consensus.cc:3037] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:38.702634 27383 raft_consensus.cc:3037] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:38.703403 27383 raft_consensus.cc:2454] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 293161cad44d4efcb40224988580abc8 in term 1.
I20260619 08:23:38.703501 27523 raft_consensus.cc:2454] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 293161cad44d4efcb40224988580abc8 in term 1.
I20260619 08:23:38.703702 27597 leader_election.cc:302] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2333a20009e84004b290128214d71e1c, 293161cad44d4efcb40224988580abc8; no voters:
I20260619 08:23:38.703824 27740 raft_consensus.cc:2781] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:38.703933 27740 raft_consensus.cc:686] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [term 1 LEADER]: Becoming Leader. State: Replica: 293161cad44d4efcb40224988580abc8, State: Running, Role: LEADER
I20260619 08:23:38.704025 27740 consensus_queue.cc:237] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } }
I20260619 08:23:38.704805 27225 catalog_manager.cc:5495] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 293161cad44d4efcb40224988580abc8 (127.25.127.131). New cstate: current_term: 1 leader_uuid: "293161cad44d4efcb40224988580abc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } health_report { overall_health: HEALTHY } } }
Jun 19 08:23:38 dist-test-slave-5rhg krb5kdc[27181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857417, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:38.763548 27768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.756815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:41381 (local address 127.0.0.1:36712)
0619 08:23:38.756957 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:38.756970 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:38.757039 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:38.757189 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:38.757189 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:38.757499 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:38.757551 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.757553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.758214 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.758215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.758753 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.758754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.759090 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.759090 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.759138 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.759618 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:38.759622 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:38.760967 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:38.762507 (+ 1540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:38.762514 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:38.762516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:38.762712 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:38.762887 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:38.762887 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:38.762887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:38.762946 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:38.763194 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:38.763200 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:38.763242 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 08:23:38.763324 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 08:23:38.763411 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":47,"threads_started":1}
I20260619 08:23:38.763721 27769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.756829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36712 (local address 127.25.127.190:41381)
0619 08:23:38.756969 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.756971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.757038 (+ 67us) server_negotiation.cc:407] Connection header received
0619 08:23:38.757089 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.757090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.757100 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.757167 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:38.757616 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.758145 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.758827 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.759022 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.759077 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.761046 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:38.761047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:38.761048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:38.761053 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:38.762396 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:38.762776 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:38.762777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:38.762777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:38.762828 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:38.762999 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:38.762999 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:38.762999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:38.763094 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:38.763174 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.763405 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.763485 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":92,"threads_started":1}
I20260619 08:23:38.765591 27225 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:36712:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 08:23:38.765812 27225 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:23:38.767387 27503 tablet_service.cc:1467] Processing CreateTablet for tablet 38fd9410af634061972f0757d0ad3794 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b93d33274e4f408285094a3f36d88148]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:23:38.767400 27643 tablet_service.cc:1467] Processing CreateTablet for tablet 38fd9410af634061972f0757d0ad3794 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b93d33274e4f408285094a3f36d88148]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:23:38.767408 27349 tablet_service.cc:1467] Processing CreateTablet for tablet 38fd9410af634061972f0757d0ad3794 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b93d33274e4f408285094a3f36d88148]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:23:38.767552 27503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38fd9410af634061972f0757d0ad3794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:38.767552 27643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38fd9410af634061972f0757d0ad3794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:38.767570 27349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38fd9410af634061972f0757d0ad3794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:38.768921 27732 tablet_bootstrap.cc:492] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c: Bootstrap starting.
I20260619 08:23:38.769063 27734 tablet_bootstrap.cc:492] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6: Bootstrap starting.
I20260619 08:23:38.769377 27734 tablet_bootstrap.cc:654] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:38.769464 27732 tablet_bootstrap.cc:654] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:38.769608 27733 tablet_bootstrap.cc:492] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8: Bootstrap starting.
I20260619 08:23:38.769927 27734 tablet_bootstrap.cc:492] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6: No bootstrap required, opened a new log
I20260619 08:23:38.769980 27734 ts_tablet_manager.cc:1397] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 08:23:38.770125 27733 tablet_bootstrap.cc:654] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:38.770164 27734 raft_consensus.cc:348] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } }
I20260619 08:23:38.770215 27732 tablet_bootstrap.cc:492] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c: No bootstrap required, opened a new log
I20260619 08:23:38.770249 27734 raft_consensus.cc:374] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:38.770279 27734 raft_consensus.cc:729] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a8d2e33a25e41e29d4937901d07e2e6, State: Initialized, Role: FOLLOWER
I20260619 08:23:38.770262 27732 ts_tablet_manager.cc:1397] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:38.770326 27734 consensus_queue.cc:260] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } }
I20260619 08:23:38.770469 27734 ts_tablet_manager.cc:1428] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:38.770514 27732 raft_consensus.cc:348] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } }
I20260619 08:23:38.770587 27732 raft_consensus.cc:374] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:38.770612 27732 raft_consensus.cc:729] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2333a20009e84004b290128214d71e1c, State: Initialized, Role: FOLLOWER
I20260619 08:23:38.770655 27732 consensus_queue.cc:260] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } }
I20260619 08:23:38.770756 27732 ts_tablet_manager.cc:1428] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:38.770944 27742 tablet_replica.cc:439] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } }
I20260619 08:23:38.771042 27733 tablet_bootstrap.cc:492] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8: No bootstrap required, opened a new log
I20260619 08:23:38.771054 27742 tablet_replica.cc:442] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:23:38.771090 27733 ts_tablet_manager.cc:1397] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 08:23:38.771226 27733 raft_consensus.cc:348] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } }
I20260619 08:23:38.771296 27733 raft_consensus.cc:374] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:38.771261 27738 tablet_replica.cc:439] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } }
I20260619 08:23:38.771327 27733 raft_consensus.cc:729] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 293161cad44d4efcb40224988580abc8, State: Initialized, Role: FOLLOWER
I20260619 08:23:38.771349 27738 tablet_replica.cc:442] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:23:38.771391 27733 consensus_queue.cc:260] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } }
I20260619 08:23:38.771483 27733 ts_tablet_manager.cc:1428] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 08:23:38.771616 27740 tablet_replica.cc:439] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } }
I20260619 08:23:38.771694 27740 tablet_replica.cc:442] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:23:38.787164 27738 raft_consensus.cc:482] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:23:38.787228 27738 raft_consensus.cc:504] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } }
I20260619 08:23:38.787539 27738 leader_election.cc:288] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a8d2e33a25e41e29d4937901d07e2e6 (127.25.127.130:40697), 293161cad44d4efcb40224988580abc8 (127.25.127.131:35615)
I20260619 08:23:38.791128 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.787679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:40697 (local address 127.25.127.129:45513)
0619 08:23:38.787840 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:38.787848 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:38.787907 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:38.788246 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:38.788247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:38.788250 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:38.788335 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.788337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.789230 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.789231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.790124 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.790127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.790850 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.790851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.790906 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.790910 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:23:38.790983 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 08:23:38.791036 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260619 08:23:38.791127 27730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.787760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:50035 (local address 127.25.127.131:35615)
0619 08:23:38.787806 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.787809 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.787915 (+ 106us) server_negotiation.cc:407] Connection header received
0619 08:23:38.787984 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.787984 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.787994 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.788049 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:38.788278 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.789142 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.790131 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.790723 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.790776 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.790783 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.790959 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.791068 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:23:38.791142 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.787712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:45513 (local address 127.25.127.130:40697)
0619 08:23:38.787755 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.787758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.787916 (+ 158us) server_negotiation.cc:407] Connection header received
0619 08:23:38.787982 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.787983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.787994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.788054 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:38.788425 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.789148 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.790220 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.790739 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.790833 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.790842 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.791013 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.791080 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:23:38.791152 27773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.787670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:35615 (local address 127.25.127.129:50035)
0619 08:23:38.787828 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:38.787840 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:38.787923 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:38.788069 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:38.788071 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:38.788072 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:38.788136 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.788138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.789271 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.789272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.790050 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.790053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.790799 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.790801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.790873 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.790878 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:23:38.790930 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:23:38.790981 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":108,"threads_started":1}
I20260619 08:23:38.791543 27663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38fd9410af634061972f0757d0ad3794" candidate_uuid: "2333a20009e84004b290128214d71e1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "293161cad44d4efcb40224988580abc8" is_pre_election: true
I20260619 08:23:38.791591 27523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38fd9410af634061972f0757d0ad3794" candidate_uuid: "2333a20009e84004b290128214d71e1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" is_pre_election: true
I20260619 08:23:38.791616 27663 raft_consensus.cc:2454] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2333a20009e84004b290128214d71e1c in term 0.
I20260619 08:23:38.791679 27523 raft_consensus.cc:2454] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2333a20009e84004b290128214d71e1c in term 0.
I20260619 08:23:38.791811 27316 leader_election.cc:302] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2333a20009e84004b290128214d71e1c, 293161cad44d4efcb40224988580abc8; no voters:
I20260619 08:23:38.791970 27738 raft_consensus.cc:2781] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:23:38.792017 27738 raft_consensus.cc:482] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:23:38.792049 27738 raft_consensus.cc:3037] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:38.792805 27738 raft_consensus.cc:504] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } }
I20260619 08:23:38.792950 27738 leader_election.cc:288] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [CANDIDATE]: Term 1 election: Requested vote from peers 9a8d2e33a25e41e29d4937901d07e2e6 (127.25.127.130:40697), 293161cad44d4efcb40224988580abc8 (127.25.127.131:35615)
I20260619 08:23:38.793083 27523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38fd9410af634061972f0757d0ad3794" candidate_uuid: "2333a20009e84004b290128214d71e1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8d2e33a25e41e29d4937901d07e2e6"
I20260619 08:23:38.793156 27523 raft_consensus.cc:3037] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:38.793740 27523 raft_consensus.cc:2454] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2333a20009e84004b290128214d71e1c in term 1.
I20260619 08:23:38.793864 27663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38fd9410af634061972f0757d0ad3794" candidate_uuid: "2333a20009e84004b290128214d71e1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "293161cad44d4efcb40224988580abc8"
I20260619 08:23:38.793916 27315 leader_election.cc:302] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2333a20009e84004b290128214d71e1c, 9a8d2e33a25e41e29d4937901d07e2e6; no voters:
I20260619 08:23:38.793936 27663 raft_consensus.cc:3037] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:38.794021 27738 raft_consensus.cc:2781] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:38.794149 27738 raft_consensus.cc:686] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [term 1 LEADER]: Becoming Leader. State: Replica: 2333a20009e84004b290128214d71e1c, State: Running, Role: LEADER
I20260619 08:23:38.794219 27738 consensus_queue.cc:237] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } }
I20260619 08:23:38.794589 27775 tablet_replica.cc:439] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c: TxnStatusTablet state changed. Reason: New leader 2333a20009e84004b290128214d71e1c. Latest consensus state: current_term: 1 leader_uuid: "2333a20009e84004b290128214d71e1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } }
I20260619 08:23:38.794667 27775 tablet_replica.cc:442] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:23:38.794674 27663 raft_consensus.cc:2454] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2333a20009e84004b290128214d71e1c in term 1.
I20260619 08:23:38.794940 27777 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 08:23:38.795056 27777 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 08:23:38.795023 27226 catalog_manager.cc:5495] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2333a20009e84004b290128214d71e1c (127.25.127.129). New cstate: current_term: 1 leader_uuid: "2333a20009e84004b290128214d71e1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } health_report { overall_health: UNKNOWN } } }
I20260619 08:23:38.801731 27768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.799007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:35459 (local address 127.0.0.1:48254)
0619 08:23:38.799081 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:38.799093 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:38.799141 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:38.799304 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:38.799305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:38.799305 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:23:38.799347 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.799348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.800218 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.800218 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.800784 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.800786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.801267 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.801267 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.801314 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.801328 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:23:38.801590 (+ 262us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:23:38.801597 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 08:23:38.801638 (+ 41us) client_negotiation.cc:240] Negotiation successful
0619 08:23:38.801677 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 08:23:38.801753 27729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.799083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48254 (local address 127.25.127.129:35459)
0619 08:23:38.799128 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.799130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.799139 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:23:38.799211 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.799212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.799229 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.799304 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:23:38.799386 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.800112 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.800886 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.801190 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.801264 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.801414 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:23:38.801525 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:23:38.801601 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.801662 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.801682 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:23:38.802973 27523 raft_consensus.cc:1261] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 1 FOLLOWER]: Refusing update from remote peer 2333a20009e84004b290128214d71e1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:38.802965 27663 raft_consensus.cc:1261] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8 [term 1 FOLLOWER]: Refusing update from remote peer 2333a20009e84004b290128214d71e1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:38.803308 27775 consensus_queue.cc:1035] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:38.803483 27738 consensus_queue.cc:1035] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:38.803982 27740 tablet_replica.cc:439] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8: TxnStatusTablet state changed. Reason: New leader 2333a20009e84004b290128214d71e1c. Latest consensus state: current_term: 1 leader_uuid: "2333a20009e84004b290128214d71e1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } }
I20260619 08:23:38.803998 27742 tablet_replica.cc:439] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6: TxnStatusTablet state changed. Reason: New leader 2333a20009e84004b290128214d71e1c. Latest consensus state: current_term: 1 leader_uuid: "2333a20009e84004b290128214d71e1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } }
I20260619 08:23:38.804097 27742 tablet_replica.cc:442] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:23:38.804097 27740 tablet_replica.cc:442] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:23:38.804904 27775 tablet_replica.cc:439] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2333a20009e84004b290128214d71e1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } }
I20260619 08:23:38.805002 27775 tablet_replica.cc:442] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:23:38.805135 27740 tablet_replica.cc:439] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2333a20009e84004b290128214d71e1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } }
I20260619 08:23:38.805226 27740 tablet_replica.cc:442] T 38fd9410af634061972f0757d0ad3794 P 293161cad44d4efcb40224988580abc8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:23:38.808918 27738 tablet_replica.cc:439] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2333a20009e84004b290128214d71e1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } }
I20260619 08:23:38.809026 27738 tablet_replica.cc:442] T 38fd9410af634061972f0757d0ad3794 P 2333a20009e84004b290128214d71e1c: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:23:38.809451 27745 mvcc.cc:205] Tried to move back new op lower bound from 7298487987414659072 to 7298487987381575680. Current Snapshot: MvccSnapshot[applied={T|T < 7298487987414659072}]
I20260619 08:23:38.809563 27742 tablet_replica.cc:439] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2333a20009e84004b290128214d71e1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 } } peers { permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 } } peers { permanent_uuid: "293161cad44d4efcb40224988580abc8" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35615 } } }
I20260619 08:23:38.809653 27730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.806660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32810 (local address 127.25.127.131:35615)
0619 08:23:38.806696 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.806698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.806725 (+ 27us) server_negotiation.cc:407] Connection header received
0619 08:23:38.806805 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.806805 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.806820 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.806873 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:23:38.806996 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.807437 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.808503 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.808841 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.808921 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.809149 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:23:38.809269 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:23:38.809332 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.809550 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.809571 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 08:23:38.809669 27742 tablet_replica.cc:442] T 38fd9410af634061972f0757d0ad3794 P 9a8d2e33a25e41e29d4937901d07e2e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:23:38.815866 27729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.812420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:50227 (local address 127.25.127.129:35459)
0619 08:23:38.812467 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.812469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.812619 (+ 150us) server_negotiation.cc:407] Connection header received
0619 08:23:38.812644 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.812645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.812653 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.812705 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:23:38.812836 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.813391 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.815106 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.815276 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.815342 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.815512 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:23:38.815556 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:23:38.815593 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.815795 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.815809 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:23:38.815932 27784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.812367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:35459 (local address 127.25.127.131:50227)
0619 08:23:38.812521 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:38.812532 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:38.812593 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:38.812726 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:38.812726 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:38.812728 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:23:38.812780 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.812781 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.813460 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.813461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.814979 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.814982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.815355 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.815356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.815422 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.815436 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:23:38.815734 (+ 298us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:23:38.815736 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:23:38.815788 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:23:38.815830 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":90,"threads_started":1}
I20260619 08:23:38.818073 27523 raft_consensus.cc:1261] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 1 FOLLOWER]: Refusing update from remote peer 293161cad44d4efcb40224988580abc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:38.818073 27383 raft_consensus.cc:1261] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c [term 1 FOLLOWER]: Refusing update from remote peer 293161cad44d4efcb40224988580abc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:38.818326 27761 consensus_queue.cc:1035] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8d2e33a25e41e29d4937901d07e2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:38.818476 27740 consensus_queue.cc:1035] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2333a20009e84004b290128214d71e1c" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:38.821354 27746 mvcc.cc:205] Tried to move back new op lower bound from 7298487987477356544 to 7298487987012407296. Current Snapshot: MvccSnapshot[applied={T|T < 7298487987477356544}]
I20260619 08:23:38.826951 27730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.824493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:41829 (local address 127.25.127.131:35615)
0619 08:23:38.824556 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.824558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.824566 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:23:38.824620 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.824621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.824630 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.824681 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:23:38.824802 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.825234 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.826115 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.826459 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.826538 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.826734 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:23:38.826766 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:23:38.826812 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.826867 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.826887 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 08:23:38.827076 27791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.824346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:35615 (local address 127.25.127.129:41829)
0619 08:23:38.824487 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:38.824497 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:38.824561 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:38.824684 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:38.824685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:38.824689 (+ 4us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:23:38.824744 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.824745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.825312 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.825312 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.826026 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:38.826029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.826557 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.826558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:38.826630 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.826643 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:23:38.826811 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:23:38.826812 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:23:38.826863 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 08:23:38.826904 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260619 08:23:38.836409 27225 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:23:38.836508 27225 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:23:38.837108 27225 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a49328ee990341c2aca22797dd19d93e: Sending DeleteTablet for 3 replicas of tablet c38923a8201142ff8a6e2cc75116d393
I20260619 08:23:38.837384 27349 tablet_service.cc:1514] Processing DeleteTablet for tablet c38923a8201142ff8a6e2cc75116d393 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:38 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:48250
I20260619 08:23:38.837482 27503 tablet_service.cc:1514] Processing DeleteTablet for tablet c38923a8201142ff8a6e2cc75116d393 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:38 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:50006
I20260619 08:23:38.837486 27755 tablet_replica.cc:330] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c: stopping tablet replica
I20260619 08:23:38.837553 27755 raft_consensus.cc:2229] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:23:38.837574 27756 tablet_replica.cc:330] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6: stopping tablet replica
I20260619 08:23:38.837596 27755 raft_consensus.cc:2258] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:38.837642 27756 raft_consensus.cc:2229] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:23:38.837684 27756 raft_consensus.cc:2258] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:38.838001 27755 ts_tablet_manager.cc:1905] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:38.838236 27643 tablet_service.cc:1514] Processing DeleteTablet for tablet c38923a8201142ff8a6e2cc75116d393 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:38 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:32804
I20260619 08:23:38.838323 27754 tablet_replica.cc:330] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8: stopping tablet replica
I20260619 08:23:38.838382 27754 raft_consensus.cc:2229] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:23:38.838469 27754 raft_consensus.cc:2258] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:38.838775 27754 ts_tablet_manager.cc:1905] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:38.839624 27755 ts_tablet_manager.cc:1918] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:23:38.839681 27755 log.cc:1196] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-0/wal/wals/c38923a8201142ff8a6e2cc75116d393
I20260619 08:23:38.839890 27755 ts_tablet_manager.cc:1939] T c38923a8201142ff8a6e2cc75116d393 P 2333a20009e84004b290128214d71e1c: Deleting consensus metadata
I20260619 08:23:38.840386 27756 ts_tablet_manager.cc:1905] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:38.840498 27754 ts_tablet_manager.cc:1918] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:23:38.840541 27754 log.cc:1196] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/c38923a8201142ff8a6e2cc75116d393
I20260619 08:23:38.840739 27754 ts_tablet_manager.cc:1939] T c38923a8201142ff8a6e2cc75116d393 P 293161cad44d4efcb40224988580abc8: Deleting consensus metadata
I20260619 08:23:38.841063 27212 catalog_manager.cc:4841] TS 293161cad44d4efcb40224988580abc8 (127.25.127.131:35615): tablet c38923a8201142ff8a6e2cc75116d393 (table test-table [id=213ce8cc5ed241ce99e5d73686342da3]) successfully deleted
I20260619 08:23:38.841799 27756 ts_tablet_manager.cc:1918] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:23:38.841837 27756 log.cc:1196] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/c38923a8201142ff8a6e2cc75116d393
I20260619 08:23:38.841981 27756 ts_tablet_manager.cc:1939] T c38923a8201142ff8a6e2cc75116d393 P 9a8d2e33a25e41e29d4937901d07e2e6: Deleting consensus metadata
I20260619 08:23:38.842347 27213 catalog_manager.cc:4841] TS 2333a20009e84004b290128214d71e1c (127.25.127.129:35459): tablet c38923a8201142ff8a6e2cc75116d393 (table test-table [id=213ce8cc5ed241ce99e5d73686342da3]) successfully deleted
I20260619 08:23:38.842468 27211 catalog_manager.cc:4841] TS 9a8d2e33a25e41e29d4937901d07e2e6 (127.25.127.130:40697): tablet c38923a8201142ff8a6e2cc75116d393 (table test-table [id=213ce8cc5ed241ce99e5d73686342da3]) successfully deleted
Jun 19 08:23:38 dist-test-slave-5rhg krb5kdc[27181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857418, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.127.129@KRBTEST.COM
I20260619 08:23:38.856885 27729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.848416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48260 (local address 127.25.127.129:35459)
0619 08:23:38.848455 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.848457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.848603 (+ 146us) server_negotiation.cc:407] Connection header received
0619 08:23:38.848671 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.848671 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.848681 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.848735 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:38.849297 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.849777 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.850523 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.850723 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.850795 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.853824 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:38.853834 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:38.853835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:38.853843 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:38.855586 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:38.855958 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:38.855961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:38.855961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:38.856020 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:38.856240 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:38.856240 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:38.856241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:38.856387 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:38.856559 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.856598 (+ 39us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.856675 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 08:23:38.857141 27423 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48260
I20260619 08:23:38.867213 27769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:38.858207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36716 (local address 127.25.127.190:41381)
0619 08:23:38.858285 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:38.858287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:38.861244 (+ 2957us) server_negotiation.cc:407] Connection header received
0619 08:23:38.861322 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:38.861323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:38.861369 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:38.861433 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:38.861944 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.862511 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.863194 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:38.863417 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:38.863513 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:38.864715 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:38.864716 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:38.864716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:38.864723 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:38.866103 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:38.866460 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:38.866462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:38.866462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:38.866519 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:38.866715 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:38.866716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:38.866717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:38.866829 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:38.866940 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:38.867028 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 08:23:38.867110 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260619 08:23:38.867532 27225 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36716
I20260619 08:23:38.868206 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 27298
I20260619 08:23:38.874451 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 27438
I20260619 08:23:38.880888 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 27578
I20260619 08:23:38.885782 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 27194
2026-06-19T08:23:38Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2449 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:23:38 dist-test-slave-5rhg krb5kdc[27800](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:23:38 dist-test-slave-5rhg krb5kdc[27800](info): set up 2 sockets
Jun 19 08:23:38 dist-test-slave-5rhg krb5kdc[27800](info): commencing operation
krb5kdc: starting...
W20260619 08:23:39.921059 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:23:39 dist-test-slave-5rhg krb5kdc[27800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:23:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:23:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:23:40.034783 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:39957
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:36063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:39957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:40.118443 27813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:40.118701 27813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:40.118738 27813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:40.119859 27813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:23:40.119894 27813 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 08:23:40.119906 27813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:40.119920 27813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:40.119931 27813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:23:40.119944 27813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:23:40.121362 27813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:36063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:39957
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:39957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:40.121551 27813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:40.121713 27813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:40.126473 27818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:40.126469 27821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:40.126531 27819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:40.126724 27813 server_base.cc:1034] running on GCE node
I20260619 08:23:40.127038 27813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:40.127357 27813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:40.128520 27813 hybrid_clock.cc:648] HybridClock initialized: now 1781857420128509 us; error 45 us; skew 500 ppm
Jun 19 08:23:40 dist-test-slave-5rhg krb5kdc[27800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857420, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:40.130970 27813 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:23:40.131448 27813 webserver.cc:460] Webserver started at http://127.25.127.190:34037/ using document root <none> and password file <none>
I20260619 08:23:40.131669 27813 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:40.131737 27813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:40.131907 27813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:40.133023 27813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "aa50403c81e64a6a87fb6ea9d49c88e7"
format_stamp: "Formatted at 2026-06-19 08:23:40 on dist-test-slave-5rhg"
server_key: "9f136c4ffe712c3c1599ac21f7da6433"
server_key_iv: "e5180e5f764915b027d940b4954a0964"
server_key_version: "encryptionkey@0"
I20260619 08:23:40.133394 27813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "aa50403c81e64a6a87fb6ea9d49c88e7"
format_stamp: "Formatted at 2026-06-19 08:23:40 on dist-test-slave-5rhg"
server_key: "9f136c4ffe712c3c1599ac21f7da6433"
server_key_iv: "e5180e5f764915b027d940b4954a0964"
server_key_version: "encryptionkey@0"
I20260619 08:23:40.134729 27813 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:23:40.135499 27828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:40.135676 27813 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:23:40.135775 27813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "aa50403c81e64a6a87fb6ea9d49c88e7"
format_stamp: "Formatted at 2026-06-19 08:23:40 on dist-test-slave-5rhg"
server_key: "9f136c4ffe712c3c1599ac21f7da6433"
server_key_iv: "e5180e5f764915b027d940b4954a0964"
server_key_version: "encryptionkey@0"
I20260619 08:23:40.135875 27813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:40.151310 27813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:40.153455 27813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:40.153626 27813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:40.158404 27813 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:39957
I20260619 08:23:40.158457 27890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:39957 every 8 connection(s)
I20260619 08:23:40.158766 27813 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:23:40.159313 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 27813
I20260619 08:23:40.159386 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:23:40.159427 27891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:40.159577 26110 external_mini_cluster.cc:1428] Setting key b5394665d45b06163fb3860bddf04e19
I20260619 08:23:40.162246 27891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7: Bootstrap starting.
I20260619 08:23:40.162860 27891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:40.163128 27891 log.cc:824] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:40.163811 27891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7: No bootstrap required, opened a new log
Jun 19 08:23:40 dist-test-slave-5rhg krb5kdc[27800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:40.166092 27891 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa50403c81e64a6a87fb6ea9d49c88e7" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39957 } }
I20260619 08:23:40.166260 27891 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:40.166307 27891 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa50403c81e64a6a87fb6ea9d49c88e7, State: Initialized, Role: FOLLOWER
I20260619 08:23:40.166446 27891 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa50403c81e64a6a87fb6ea9d49c88e7" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39957 } }
I20260619 08:23:40.166535 27891 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:23:40.166568 27891 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:23:40.166654 27891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:40.167363 27891 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa50403c81e64a6a87fb6ea9d49c88e7" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39957 } }
I20260619 08:23:40.167483 27891 leader_election.cc:302] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa50403c81e64a6a87fb6ea9d49c88e7; no voters:
I20260619 08:23:40.167644 27891 leader_election.cc:288] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:23:40.167716 27896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:40.167919 27891 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:40.167934 27896 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7 [term 1 LEADER]: Becoming Leader. State: Replica: aa50403c81e64a6a87fb6ea9d49c88e7, State: Running, Role: LEADER
I20260619 08:23:40.168210 27896 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa50403c81e64a6a87fb6ea9d49c88e7" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39957 } }
I20260619 08:23:40.168566 27898 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa50403c81e64a6a87fb6ea9d49c88e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa50403c81e64a6a87fb6ea9d49c88e7" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39957 } } }
I20260619 08:23:40.168653 27898 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7 [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:40.168553 27896 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa50403c81e64a6a87fb6ea9d49c88e7. Latest consensus state: current_term: 1 leader_uuid: "aa50403c81e64a6a87fb6ea9d49c88e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa50403c81e64a6a87fb6ea9d49c88e7" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39957 } } }
I20260619 08:23:40.168965 27896 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7 [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:40.169452 27905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:23:40.169991 27905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:23:40.170816 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.160132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42010 (local address 127.25.127.190:39957)
0619 08:23:40.160328 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.160338 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.160372 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:23:40.160836 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.160849 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.160896 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.161064 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:40.161687 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.162668 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.163521 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.164001 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.164120 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.166669 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:40.166673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:40.166674 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:40.166685 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:40.168728 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:40.169158 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:40.169162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:40.169163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:40.169276 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:40.169510 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:40.169511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:40.169511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:40.169854 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:40.170022 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.170467 (+ 445us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.170603 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
I20260619 08:23:40.171711 27905 catalog_manager.cc:1347] Generated new cluster ID: 17e7181dd3da491da90db670ca6c840d
I20260619 08:23:40.171761 27905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:23:40.179476 27905 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:23:40.180217 27905 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:23:40.184739 27905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7: Generated new TSK 0
I20260619 08:23:40.184909 27905 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
I20260619 08:23:40.211686 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:39957
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:36063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:40.293223 27917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:40.293429 27917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:40.293455 27917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:40.294538 27917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:40.294572 27917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:40.294593 27917 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 08:23:40.294605 27917 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:23:40.294622 27917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:23:40.296082 27917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:36063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.127.190:39957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:40.296294 27917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:40.296471 27917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:23:40.301700 27917 server_base.cc:1034] running on GCE node
W20260619 08:23:40.301683 27923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:40.301687 27922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:40.301679 27925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:40.302115 27917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:40.302388 27917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:40.303558 27917 hybrid_clock.cc:648] HybridClock initialized: now 1781857420303537 us; error 35 us; skew 500 ppm
Jun 19 08:23:40 dist-test-slave-5rhg krb5kdc[27800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857420, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:40.306018 27917 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:23:40.306529 27917 webserver.cc:460] Webserver started at http://127.25.127.129:43139/ using document root <none> and password file <none>
I20260619 08:23:40.306757 27917 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:40.306825 27917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:40.306995 27917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:40.308046 27917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "7d5c92e30c474e38baf39c67d696d4c5"
format_stamp: "Formatted at 2026-06-19 08:23:40 on dist-test-slave-5rhg"
server_key: "d274b5ec5c80e0ec92c5573c75eab740"
server_key_iv: "fc00c0d414bb0bf2a21e029e04c05e7f"
server_key_version: "encryptionkey@0"
I20260619 08:23:40.308482 27917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "7d5c92e30c474e38baf39c67d696d4c5"
format_stamp: "Formatted at 2026-06-19 08:23:40 on dist-test-slave-5rhg"
server_key: "d274b5ec5c80e0ec92c5573c75eab740"
server_key_iv: "fc00c0d414bb0bf2a21e029e04c05e7f"
server_key_version: "encryptionkey@0"
I20260619 08:23:40.309815 27917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:40.310606 27932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:40.310774 27917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:23:40.310890 27917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "7d5c92e30c474e38baf39c67d696d4c5"
format_stamp: "Formatted at 2026-06-19 08:23:40 on dist-test-slave-5rhg"
server_key: "d274b5ec5c80e0ec92c5573c75eab740"
server_key_iv: "fc00c0d414bb0bf2a21e029e04c05e7f"
server_key_version: "encryptionkey@0"
I20260619 08:23:40.310967 27917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:40.319989 27917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:40.322135 27917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:40.322305 27917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:40.322880 27917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:40.322966 27917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:40.323031 27917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:40.323062 27917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:40.330565 27917 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:35327
I20260619 08:23:40.330654 28047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:35327 every 8 connection(s)
Jun 19 08:23:40 dist-test-slave-5rhg krb5kdc[27800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857420, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:40.330991 27917 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:23:40.334183 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.323460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:34089 (local address 127.25.127.190:39957)
0619 08:23:40.323530 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.323533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.323798 (+ 265us) server_negotiation.cc:407] Connection header received
0619 08:23:40.324576 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.324577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.324591 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.324657 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:40.325592 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.326113 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.327855 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.328089 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.328183 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.331461 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:40.331462 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:40.331462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:40.331469 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:40.332982 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:40.333369 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:40.333369 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:40.333370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:40.333419 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:40.333631 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:40.333632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:40.333632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:40.333802 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:40.333857 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.334059 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.334123 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:23:40.334390 27941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.323449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:39957 (local address 127.25.127.129:34089)
0619 08:23:40.323649 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:40.323701 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:40.324456 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:40.324737 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:40.324742 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:40.325151 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:40.325475 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.325483 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.326219 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.326221 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.327758 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.327761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.328180 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.328181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.328347 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.329516 (+ 1169us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:40.329527 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:40.331348 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:40.333052 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:40.333059 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:40.333060 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:40.333302 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:40.333476 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:40.333476 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:40.333477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:40.333568 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:40.333860 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:40.333864 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:40.333970 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 08:23:40.334048 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 08:23:40.334164 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":105,"threads_started":1}
I20260619 08:23:40.334939 28048 heartbeater.cc:347] Connected to a master server at 127.25.127.190:39957
I20260619 08:23:40.335034 28048 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:40.335201 28048 heartbeater.cc:511] Master 127.25.127.190:39957 requested a full tablet report, sending...
I20260619 08:23:40.335731 27844 ts_manager.cc:194] Registered new tserver with Master: 7d5c92e30c474e38baf39c67d696d4c5 (127.25.127.129:35327)
I20260619 08:23:40.336555 27844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:34089
I20260619 08:23:40.336550 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 27917
I20260619 08:23:40.336673 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:23:40.336854 26110 external_mini_cluster.cc:1428] Setting key f85e9fc676aacac6b8ef7d165fc09d6a
I20260619 08:23:40.342470 28054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.335921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:39957 (local address 127.25.127.129:50431)
0619 08:23:40.336073 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:40.336081 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:40.336167 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:40.336310 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:40.336311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:40.336519 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:40.336592 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.336594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.337597 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.337598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.338234 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.338236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.338736 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.338737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.338802 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.339279 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:40.339286 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:40.339686 (+ 400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:40.341510 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:40.341511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:40.341511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:40.341670 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:40.341883 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:40.341883 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:40.341884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:40.341931 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:40.342184 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:40.342185 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:40.342229 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 08:23:40.342277 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 08:23:40.342379 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260619 08:23:40.342567 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.335977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:50431 (local address 127.25.127.190:39957)
0619 08:23:40.336011 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.336013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.336174 (+ 161us) server_negotiation.cc:407] Connection header received
0619 08:23:40.336244 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.336245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.336260 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.336308 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:40.336688 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.337505 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.338324 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.338634 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.338704 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.339792 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:40.339794 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:40.339794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:40.339801 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:40.341420 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:40.341739 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:40.341740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:40.341740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:40.341805 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:40.341987 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:40.341988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:40.341988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:40.342123 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:40.342191 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.342329 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.342431 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":148,"server-negotiator.queue_time_us":17}
WARNING: no policy specified for kudu/127.25.127.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
I20260619 08:23:40.360960 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:39957
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:36063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:40.443828 28057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:40.444051 28057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:40.444089 28057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:40.445324 28057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:40.445389 28057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:40.445427 28057 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 08:23:40.445473 28057 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:23:40.445502 28057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:23:40.446959 28057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:36063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.127.190:39957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:40.447217 28057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:40.447438 28057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:40.452380 28062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:40.452376 28065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:40.452440 28057 server_base.cc:1034] running on GCE node
W20260619 08:23:40.452380 28063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:40.452852 28057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:40.453111 28057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:40.454275 28057 hybrid_clock.cc:648] HybridClock initialized: now 1781857420454263 us; error 42 us; skew 500 ppm
Jun 19 08:23:40 dist-test-slave-5rhg krb5kdc[27800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857420, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:40.456851 28057 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:23:40.457312 28057 webserver.cc:460] Webserver started at http://127.25.127.130:44019/ using document root <none> and password file <none>
I20260619 08:23:40.457548 28057 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:40.457615 28057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:40.457762 28057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:40.458810 28057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6"
format_stamp: "Formatted at 2026-06-19 08:23:40 on dist-test-slave-5rhg"
server_key: "8f15b36d8bf868704b15d02edb19326b"
server_key_iv: "b9df3aa2afd01e744bf7a27a03a5240d"
server_key_version: "encryptionkey@0"
I20260619 08:23:40.459175 28057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6"
format_stamp: "Formatted at 2026-06-19 08:23:40 on dist-test-slave-5rhg"
server_key: "8f15b36d8bf868704b15d02edb19326b"
server_key_iv: "b9df3aa2afd01e744bf7a27a03a5240d"
server_key_version: "encryptionkey@0"
I20260619 08:23:40.460534 28057 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:23:40.461251 28072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:40.461427 28057 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:23:40.461530 28057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6"
format_stamp: "Formatted at 2026-06-19 08:23:40 on dist-test-slave-5rhg"
server_key: "8f15b36d8bf868704b15d02edb19326b"
server_key_iv: "b9df3aa2afd01e744bf7a27a03a5240d"
server_key_version: "encryptionkey@0"
I20260619 08:23:40.461601 28057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:40.487658 28057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:40.490132 28057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:40.490304 28057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:40.490866 28057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:40.490931 28057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:40.490988 28057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:40.491037 28057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 08:23:40 dist-test-slave-5rhg krb5kdc[27800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857420, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:40.498453 28057 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:37517
I20260619 08:23:40.498564 28187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:37517 every 8 connection(s)
I20260619 08:23:40.498934 28057 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:23:40.500643 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.491476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:55955 (local address 127.25.127.190:39957)
0619 08:23:40.491562 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.491565 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.491713 (+ 148us) server_negotiation.cc:407] Connection header received
0619 08:23:40.492249 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.492250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.492264 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.492349 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:40.493013 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.493851 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.494909 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.495256 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.495337 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.497821 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:40.497822 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:40.497823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:40.497831 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:40.499500 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:40.499838 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:40.499839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:40.499839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:40.499889 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:40.500075 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:40.500076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:40.500076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:40.500247 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:40.500298 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.500520 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.500579 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 08:23:40.500856 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.491384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:39957 (local address 127.25.127.130:55955)
0619 08:23:40.491579 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:40.491619 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:40.492107 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:40.492439 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:40.492443 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:40.492742 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:40.492937 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.492941 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.493960 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.493961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.494841 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.494843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.495352 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.495353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.495517 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.496069 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:40.496077 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:40.497715 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:40.499561 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:40.499564 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:40.499565 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:40.499770 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:40.499931 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:40.499931 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:40.499932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:40.500014 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:40.500296 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:40.500299 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:40.500414 (+ 115us) client_negotiation.cc:769] Sending connection context
0619 08:23:40.500496 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 08:23:40.500615 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260619 08:23:40.501385 28188 heartbeater.cc:347] Connected to a master server at 127.25.127.190:39957
I20260619 08:23:40.501467 28188 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:40.501643 28188 heartbeater.cc:511] Master 127.25.127.190:39957 requested a full tablet report, sending...
I20260619 08:23:40.501997 27845 ts_manager.cc:194] Registered new tserver with Master: d85bad83c1ba4bcab1d6055ee0acf2e6 (127.25.127.130:37517)
I20260619 08:23:40.502518 27845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:55955
I20260619 08:23:40.506060 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 28057
I20260619 08:23:40.506181 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:23:40.506374 26110 external_mini_cluster.cc:1428] Setting key a53f9947a1d2425a613ffa04f1331841
I20260619 08:23:40.507870 28194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.502392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:39957 (local address 127.25.127.130:49555)
0619 08:23:40.502562 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:40.502574 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:40.502651 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:40.502785 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:40.502785 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:40.502938 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:40.502992 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.502993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.503631 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.503632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.504139 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.504141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.504536 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.504537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.504622 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.504972 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:40.504976 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:40.505235 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:40.506720 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:40.506721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:40.506722 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:40.506926 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:40.507165 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:40.507165 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:40.507166 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:40.507234 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:40.507526 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:40.507528 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:40.507575 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 08:23:40.507658 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 08:23:40.507763 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260619 08:23:40.507864 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.502489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:49555 (local address 127.25.127.190:39957)
0619 08:23:40.502530 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.502532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.502626 (+ 94us) server_negotiation.cc:407] Connection header received
0619 08:23:40.502707 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.502708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.502718 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.502773 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:40.503071 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.503536 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.504248 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.504461 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.504537 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.505324 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:40.505325 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:40.505326 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:40.505333 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:40.506638 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:40.507002 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:40.507003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:40.507004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:40.507079 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:40.507297 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:40.507298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:40.507298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:40.507448 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:40.507524 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.507684 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.507762 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for kudu/127.25.127.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
I20260619 08:23:40.529201 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:39957
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:36063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:40.614681 28197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:40.614909 28197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:40.614935 28197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:40.616115 28197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:40.616191 28197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:40.616216 28197 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 08:23:40.616281 28197 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:23:40.616314 28197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:23:40.617791 28197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:36063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.127.190:39957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:40.618055 28197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:40.618297 28197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:40.623521 28203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:40.623507 28205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:40.623525 28197 server_base.cc:1034] running on GCE node
W20260619 08:23:40.623507 28202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:40.624022 28197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:40.624315 28197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:40.625478 28197 hybrid_clock.cc:648] HybridClock initialized: now 1781857420625465 us; error 27 us; skew 500 ppm
Jun 19 08:23:40 dist-test-slave-5rhg krb5kdc[27800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857420, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:40.628309 28197 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:23:40.628814 28197 webserver.cc:460] Webserver started at http://127.25.127.131:38393/ using document root <none> and password file <none>
I20260619 08:23:40.629053 28197 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:40.629098 28197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:40.629261 28197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:40.630311 28197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "8e267a01bb5645abb97cad8408953ed3"
format_stamp: "Formatted at 2026-06-19 08:23:40 on dist-test-slave-5rhg"
server_key: "b58c77612e8ca6bd0d58bdf2ef02e2c0"
server_key_iv: "74d6d086065427e8f08207d5651ef399"
server_key_version: "encryptionkey@0"
I20260619 08:23:40.630671 28197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "8e267a01bb5645abb97cad8408953ed3"
format_stamp: "Formatted at 2026-06-19 08:23:40 on dist-test-slave-5rhg"
server_key: "b58c77612e8ca6bd0d58bdf2ef02e2c0"
server_key_iv: "74d6d086065427e8f08207d5651ef399"
server_key_version: "encryptionkey@0"
I20260619 08:23:40.631970 28197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:23:40.632800 28212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:40.633006 28197 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:40.633098 28197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "8e267a01bb5645abb97cad8408953ed3"
format_stamp: "Formatted at 2026-06-19 08:23:40 on dist-test-slave-5rhg"
server_key: "b58c77612e8ca6bd0d58bdf2ef02e2c0"
server_key_iv: "74d6d086065427e8f08207d5651ef399"
server_key_version: "encryptionkey@0"
I20260619 08:23:40.633168 28197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:40.656368 28197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:40.658288 28197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:40.658445 28197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:40.659085 28197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:40.659127 28197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:40.659152 28197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:40.659163 28197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 08:23:40 dist-test-slave-5rhg krb5kdc[27800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857420, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:40.666929 28197 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:45725
I20260619 08:23:40.667032 28327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:45725 every 8 connection(s)
I20260619 08:23:40.667338 28197 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:23:40.669771 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.659538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:51467 (local address 127.25.127.190:39957)
0619 08:23:40.659609 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.659611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.659800 (+ 189us) server_negotiation.cc:407] Connection header received
0619 08:23:40.660359 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.660361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.660375 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.660457 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:40.661134 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.661988 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.663054 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.663265 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.663352 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.666229 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:40.666231 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:40.666232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:40.666240 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:40.668270 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:40.668803 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:40.668804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:40.668805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:40.668872 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:40.669180 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:40.669181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:40.669182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:40.669313 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:40.669398 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.669640 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.669708 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 08:23:40.669973 28221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.659520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:39957 (local address 127.25.127.131:51467)
0619 08:23:40.659668 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:40.659709 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:40.660264 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:40.660521 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:40.660524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:40.660806 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:40.661035 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.661040 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.662084 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.662085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.662943 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.662945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.663366 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.663367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.663509 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.664193 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:40.664212 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:40.666126 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:40.668372 (+ 2246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:40.668380 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:40.668382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:40.668684 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:40.668968 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:40.668969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:40.668970 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:40.669093 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:40.669397 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:40.669402 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:40.669523 (+ 121us) client_negotiation.cc:769] Sending connection context
0619 08:23:40.669607 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 08:23:40.669733 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":83,"threads_started":1}
I20260619 08:23:40.670519 28328 heartbeater.cc:347] Connected to a master server at 127.25.127.190:39957
I20260619 08:23:40.670629 28328 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:40.670838 28328 heartbeater.cc:511] Master 127.25.127.190:39957 requested a full tablet report, sending...
I20260619 08:23:40.671254 27844 ts_manager.cc:194] Registered new tserver with Master: 8e267a01bb5645abb97cad8408953ed3 (127.25.127.131:45725)
I20260619 08:23:40.671679 27844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:51467
I20260619 08:23:40.674033 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 28197
I20260619 08:23:40.674134 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:23:40.674295 26110 external_mini_cluster.cc:1428] Setting key 9fa65d4b04a68c97277297d8c528c8ea
I20260619 08:23:40.675205 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:23:40.677639 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.671808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:55509 (local address 127.25.127.190:39957)
0619 08:23:40.671843 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.671847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.672020 (+ 173us) server_negotiation.cc:407] Connection header received
0619 08:23:40.672060 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.672061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.672071 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.672128 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:40.672525 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.672981 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.673643 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.673811 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.673881 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.674885 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:40.674886 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:40.674886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:40.674893 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:40.676536 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:40.676859 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:40.676860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:40.676860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:40.676922 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:40.677108 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:40.677108 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:40.677109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:40.677267 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:40.677341 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.677523 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.677581 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:23:40.677733 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.671693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:39957 (local address 127.25.127.131:55509)
0619 08:23:40.671954 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:40.671964 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:40.672012 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:40.672169 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:40.672170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:40.672373 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:40.672450 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.672452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.673066 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.673066 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.673578 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.673579 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.673881 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.673881 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.673940 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.674432 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:40.674439 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:40.674813 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:40.676628 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:40.676629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:40.676629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:40.676792 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:40.676987 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:40.676987 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:40.676987 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:40.677038 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:40.677346 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:40.677347 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:40.677406 (+ 59us) client_negotiation.cc:769] Sending connection context
0619 08:23:40.677485 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 08:23:40.677604 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":76,"threads_started":1}
Jun 19 08:23:40 dist-test-slave-5rhg krb5kdc[27800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 08:23:40 dist-test-slave-5rhg krb5kdc[27800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:40.698396 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.691374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42022 (local address 127.25.127.190:39957)
0619 08:23:40.691439 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.691441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.691452 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:23:40.691513 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.691514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.691524 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.691574 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:40.692118 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.692650 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.693393 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.693577 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.693632 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.695779 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:40.695780 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:40.695781 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:40.695788 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:40.697250 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:40.697618 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:40.697619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:40.697619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:40.697678 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:40.697881 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:40.697882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:40.697882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:40.697992 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:40.698095 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.698207 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.698288 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:23:40.699972 27844 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:23:40.701746 27844 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:23:40.708547 28348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.704064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:35327 (local address 127.0.0.1:50742)
0619 08:23:40.704457 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:40.704468 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:40.704530 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:40.704921 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:40.704921 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:40.704923 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:40.704988 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.704990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.706283 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.706284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.707128 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.707131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.708196 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.708198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.708305 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.708324 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 08:23:40.708406 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 08:23:40.708468 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":20,"thread_start_us":107,"threads_started":1}
I20260619 08:23:40.708571 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.704193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50742 (local address 127.25.127.129:35327)
0619 08:23:40.704401 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.704408 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.704522 (+ 114us) server_negotiation.cc:407] Connection header received
0619 08:23:40.704600 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.704602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.704618 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.704724 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:40.705103 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.706182 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.707214 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.708075 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.708194 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.708257 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.708428 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.708471 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":72,"threads_started":1}
I20260619 08:23:40.708950 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.704351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35434 (local address 127.25.127.131:45725)
0619 08:23:40.704515 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.704520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.704705 (+ 185us) server_negotiation.cc:407] Connection header received
0619 08:23:40.704756 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.704758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.704772 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.704844 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:40.705165 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.705856 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.707369 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.708097 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.708194 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.708257 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.708469 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.708518 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
I20260619 08:23:40.709179 28345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.704254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:45725 (local address 127.0.0.1:35434)
0619 08:23:40.704619 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:40.704629 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:40.704694 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:40.705070 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:40.705071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:40.705073 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:40.705108 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.705109 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.705954 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.705955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.707237 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.707240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.708198 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.708199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.708305 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.708324 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 08:23:40.709072 (+ 748us) client_negotiation.cc:240] Negotiation successful
0619 08:23:40.709120 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":34,"thread_start_us":68,"threads_started":1}
I20260619 08:23:40.709378 28346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.704048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:37517 (local address 127.0.0.1:41186)
0619 08:23:40.704221 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:40.704264 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:40.704328 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:40.704770 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:40.704773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:40.704774 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:40.704843 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.704845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.706209 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.706210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.707272 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.707274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.709150 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.709151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.709218 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.709224 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 08:23:40.709259 (+ 35us) client_negotiation.cc:240] Negotiation successful
0619 08:23:40.709307 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":47,"threads_started":1}
I20260619 08:23:40.709339 27973 tablet_service.cc:1467] Processing CreateTablet for tablet 79c80d8e51dd4e0f8cef64535d482cf2 (DEFAULT_TABLE table=test-table [id=7c79cc6293ad4f51866b351b2d2d1086]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:40.709760 28350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.704444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41186 (local address 127.25.127.130:37517)
0619 08:23:40.704586 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.704590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.704601 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:23:40.704643 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.704646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.704661 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.704766 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:40.704974 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.706123 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.708295 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.709042 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.709554 (+ 512us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.709624 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.709649 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.709687 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260619 08:23:40.709751 27973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79c80d8e51dd4e0f8cef64535d482cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:40.709842 28262 tablet_service.cc:1467] Processing CreateTablet for tablet 79c80d8e51dd4e0f8cef64535d482cf2 (DEFAULT_TABLE table=test-table [id=7c79cc6293ad4f51866b351b2d2d1086]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:40.710170 28262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79c80d8e51dd4e0f8cef64535d482cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:40.710366 28122 tablet_service.cc:1467] Processing CreateTablet for tablet 79c80d8e51dd4e0f8cef64535d482cf2 (DEFAULT_TABLE table=test-table [id=7c79cc6293ad4f51866b351b2d2d1086]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:40.710660 28122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79c80d8e51dd4e0f8cef64535d482cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:40.713429 28351 tablet_bootstrap.cc:492] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3: Bootstrap starting.
I20260619 08:23:40.713783 28353 tablet_bootstrap.cc:492] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6: Bootstrap starting.
I20260619 08:23:40.713841 28352 tablet_bootstrap.cc:492] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5: Bootstrap starting.
I20260619 08:23:40.714421 28352 tablet_bootstrap.cc:654] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:40.714501 28353 tablet_bootstrap.cc:654] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:40.714660 28351 tablet_bootstrap.cc:654] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:40.714771 28352 log.cc:824] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:40.714929 28353 log.cc:824] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:40.714984 28351 log.cc:824] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:40.715577 28352 tablet_bootstrap.cc:492] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5: No bootstrap required, opened a new log
I20260619 08:23:40.715629 28352 ts_tablet_manager.cc:1397] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:23:40.715680 28351 tablet_bootstrap.cc:492] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3: No bootstrap required, opened a new log
I20260619 08:23:40.715680 28353 tablet_bootstrap.cc:492] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6: No bootstrap required, opened a new log
I20260619 08:23:40.715739 28353 ts_tablet_manager.cc:1397] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:23:40.715739 28351 ts_tablet_manager.cc:1397] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:23:40.717216 28352 raft_consensus.cc:348] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } }
I20260619 08:23:40.717344 28352 raft_consensus.cc:374] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:40.717367 28352 raft_consensus.cc:729] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d5c92e30c474e38baf39c67d696d4c5, State: Initialized, Role: FOLLOWER
I20260619 08:23:40.717483 28352 consensus_queue.cc:260] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } }
I20260619 08:23:40.717736 28352 ts_tablet_manager.cc:1428] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:23:40.717746 28048 heartbeater.cc:503] Master 127.25.127.190:39957 was elected leader, sending a full tablet report...
I20260619 08:23:40.717768 28353 raft_consensus.cc:348] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } }
I20260619 08:23:40.717919 28353 raft_consensus.cc:374] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:40.717957 28353 raft_consensus.cc:729] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d85bad83c1ba4bcab1d6055ee0acf2e6, State: Initialized, Role: FOLLOWER
I20260619 08:23:40.717988 28351 raft_consensus.cc:348] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } }
I20260619 08:23:40.718070 28353 consensus_queue.cc:260] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } }
I20260619 08:23:40.718119 28351 raft_consensus.cc:374] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:40.718154 28351 raft_consensus.cc:729] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e267a01bb5645abb97cad8408953ed3, State: Initialized, Role: FOLLOWER
I20260619 08:23:40.718259 28351 consensus_queue.cc:260] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } }
I20260619 08:23:40.718319 28353 ts_tablet_manager.cc:1428] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:23:40.718389 28188 heartbeater.cc:503] Master 127.25.127.190:39957 was elected leader, sending a full tablet report...
I20260619 08:23:40.718482 28351 ts_tablet_manager.cc:1428] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:23:40.718641 28328 heartbeater.cc:503] Master 127.25.127.190:39957 was elected leader, sending a full tablet report...
I20260619 08:23:40.741451 28358 raft_consensus.cc:482] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:23:40.741567 28358 raft_consensus.cc:504] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } }
I20260619 08:23:40.742067 28358 leader_election.cc:288] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e267a01bb5645abb97cad8408953ed3 (127.25.127.131:45725), 7d5c92e30c474e38baf39c67d696d4c5 (127.25.127.129:35327)
I20260619 08:23:40.745437 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.742371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:54773 (local address 127.25.127.129:35327)
0619 08:23:40.742418 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.742420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.742432 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:23:40.742459 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.742460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.742477 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.742556 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:40.742731 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.743522 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.744656 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.745048 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.745103 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.745110 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.745305 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.745358 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:23:40.745476 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.742254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:35327 (local address 127.25.127.130:54773)
0619 08:23:40.742293 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:40.742306 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:40.742347 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:40.742569 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:40.742569 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:40.742571 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:40.742656 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.742658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.743607 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.743608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.744568 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.744570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.745126 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.745128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.745213 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.745216 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:23:40.745268 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:23:40.745300 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 08:23:40.745498 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.742358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:59219 (local address 127.25.127.131:45725)
0619 08:23:40.742424 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.742426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.742530 (+ 104us) server_negotiation.cc:407] Connection header received
0619 08:23:40.742558 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.742559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.742567 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.742626 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:40.742802 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.743541 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.744702 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.745074 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.745168 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.745178 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.745374 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.745431 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:23:40.745498 28360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.742254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:45725 (local address 127.25.127.130:59219)
0619 08:23:40.742459 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:40.742475 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:40.742514 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:40.742647 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:40.742648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:40.742650 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:40.742721 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.742722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.743634 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.743635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.744593 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.744595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.745182 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.745183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.745286 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.745290 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:23:40.745347 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 08:23:40.745382 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":45,"thread_start_us":114,"threads_started":1}
I20260619 08:23:40.745801 28002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79c80d8e51dd4e0f8cef64535d482cf2" candidate_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d5c92e30c474e38baf39c67d696d4c5" is_pre_election: true
I20260619 08:23:40.745857 28282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79c80d8e51dd4e0f8cef64535d482cf2" candidate_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e267a01bb5645abb97cad8408953ed3" is_pre_election: true
I20260619 08:23:40.745981 28002 raft_consensus.cc:2454] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d85bad83c1ba4bcab1d6055ee0acf2e6 in term 0.
I20260619 08:23:40.746016 28282 raft_consensus.cc:2454] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d85bad83c1ba4bcab1d6055ee0acf2e6 in term 0.
I20260619 08:23:40.746204 28073 leader_election.cc:302] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d5c92e30c474e38baf39c67d696d4c5, d85bad83c1ba4bcab1d6055ee0acf2e6; no voters:
I20260619 08:23:40.746373 28358 raft_consensus.cc:2781] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:23:40.746454 28358 raft_consensus.cc:482] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:23:40.746485 28358 raft_consensus.cc:3037] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:40.747200 28358 raft_consensus.cc:504] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } }
I20260619 08:23:40.747352 28358 leader_election.cc:288] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [CANDIDATE]: Term 1 election: Requested vote from peers 8e267a01bb5645abb97cad8408953ed3 (127.25.127.131:45725), 7d5c92e30c474e38baf39c67d696d4c5 (127.25.127.129:35327)
I20260619 08:23:40.747495 28002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79c80d8e51dd4e0f8cef64535d482cf2" candidate_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d5c92e30c474e38baf39c67d696d4c5"
I20260619 08:23:40.747496 28282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79c80d8e51dd4e0f8cef64535d482cf2" candidate_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e267a01bb5645abb97cad8408953ed3"
I20260619 08:23:40.747571 28002 raft_consensus.cc:3037] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:40.747571 28282 raft_consensus.cc:3037] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:40.748466 28002 raft_consensus.cc:2454] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d85bad83c1ba4bcab1d6055ee0acf2e6 in term 1.
I20260619 08:23:40.748466 28282 raft_consensus.cc:2454] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d85bad83c1ba4bcab1d6055ee0acf2e6 in term 1.
I20260619 08:23:40.748648 28073 leader_election.cc:302] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d5c92e30c474e38baf39c67d696d4c5, d85bad83c1ba4bcab1d6055ee0acf2e6; no voters:
I20260619 08:23:40.748744 28358 raft_consensus.cc:2781] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:40.748929 28358 raft_consensus.cc:686] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 LEADER]: Becoming Leader. State: Replica: d85bad83c1ba4bcab1d6055ee0acf2e6, State: Running, Role: LEADER
I20260619 08:23:40.749034 28358 consensus_queue.cc:237] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } }
W20260619 08:23:40.749809 28189 tablet.cc:2369] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:23:40.749910 27845 catalog_manager.cc:5495] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to d85bad83c1ba4bcab1d6055ee0acf2e6 (127.25.127.130). New cstate: current_term: 1 leader_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } health_report { overall_health: UNKNOWN } } }
Jun 19 08:23:40 dist-test-slave-5rhg krb5kdc[27800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857420, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:40.779726 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.772946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42028 (local address 127.25.127.190:39957)
0619 08:23:40.773031 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.773034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.773153 (+ 119us) server_negotiation.cc:407] Connection header received
0619 08:23:40.773254 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.773255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.773270 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.773327 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:40.773799 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.774287 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.775005 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.775189 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.775285 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.777373 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:40.777374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:40.777375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:40.777381 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:40.778738 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:40.779082 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:40.779082 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:40.779083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:40.779131 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:40.779295 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:40.779296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:40.779296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:40.779420 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:40.779471 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.779590 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.779660 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":44}
I20260619 08:23:40.779832 28368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.772882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:39957 (local address 127.0.0.1:42028)
0619 08:23:40.773078 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:40.773088 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:40.773170 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:40.773331 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:40.773332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:40.773663 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:40.773710 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.773712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.774384 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.774385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.774917 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.774918 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.775284 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.775285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.775338 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.775891 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:40.775896 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:40.777287 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:40.778809 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:40.778816 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:40.778817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:40.779017 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:40.779186 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:40.779187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:40.779187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:40.779243 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:40.779473 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:40.779479 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:40.779522 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 08:23:40.779574 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:23:40.779726 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":63,"thread_start_us":118,"threads_started":1}
I20260619 08:23:40.781993 27845 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:42028:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 08:23:40.782199 27845 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:23:40.783768 28262 tablet_service.cc:1467] Processing CreateTablet for tablet 1db91b021f744370b62ebba8c157c829 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=94f242bc7ce04eb6a9cf872b02685c32]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:23:40.783753 28122 tablet_service.cc:1467] Processing CreateTablet for tablet 1db91b021f744370b62ebba8c157c829 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=94f242bc7ce04eb6a9cf872b02685c32]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:23:40.783769 27973 tablet_service.cc:1467] Processing CreateTablet for tablet 1db91b021f744370b62ebba8c157c829 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=94f242bc7ce04eb6a9cf872b02685c32]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:23:40.783946 27973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1db91b021f744370b62ebba8c157c829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:40.783946 28122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1db91b021f744370b62ebba8c157c829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:40.783946 28262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1db91b021f744370b62ebba8c157c829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:40.785276 28351 tablet_bootstrap.cc:492] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3: Bootstrap starting.
I20260619 08:23:40.785493 28352 tablet_bootstrap.cc:492] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5: Bootstrap starting.
I20260619 08:23:40.785493 28353 tablet_bootstrap.cc:492] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6: Bootstrap starting.
I20260619 08:23:40.785804 28351 tablet_bootstrap.cc:654] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:40.786098 28353 tablet_bootstrap.cc:654] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:40.786207 28352 tablet_bootstrap.cc:654] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:40.786834 28353 tablet_bootstrap.cc:492] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6: No bootstrap required, opened a new log
I20260619 08:23:40.786901 28353 ts_tablet_manager.cc:1397] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:40.787079 28351 tablet_bootstrap.cc:492] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3: No bootstrap required, opened a new log
I20260619 08:23:40.787056 28353 raft_consensus.cc:348] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:40.787115 28351 ts_tablet_manager.cc:1397] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:23:40.787122 28353 raft_consensus.cc:374] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:40.787137 28353 raft_consensus.cc:729] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d85bad83c1ba4bcab1d6055ee0acf2e6, State: Initialized, Role: FOLLOWER
I20260619 08:23:40.787173 28353 consensus_queue.cc:260] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:40.787264 28353 ts_tablet_manager.cc:1428] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:40.787276 28351 raft_consensus.cc:348] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:40.787340 28351 raft_consensus.cc:374] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:40.787360 28351 raft_consensus.cc:729] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e267a01bb5645abb97cad8408953ed3, State: Initialized, Role: FOLLOWER
I20260619 08:23:40.787321 28358 tablet_replica.cc:439] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } }
I20260619 08:23:40.787413 28358 tablet_replica.cc:442] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:23:40.787401 28351 consensus_queue.cc:260] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:40.787498 28351 ts_tablet_manager.cc:1428] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:40.787518 28352 tablet_bootstrap.cc:492] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5: No bootstrap required, opened a new log
I20260619 08:23:40.787557 28352 ts_tablet_manager.cc:1397] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:23:40.787655 28359 tablet_replica.cc:439] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } }
I20260619 08:23:40.787703 28352 raft_consensus.cc:348] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:40.787737 28359 tablet_replica.cc:442] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:23:40.787772 28352 raft_consensus.cc:374] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:40.787797 28352 raft_consensus.cc:729] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d5c92e30c474e38baf39c67d696d4c5, State: Initialized, Role: FOLLOWER
I20260619 08:23:40.787837 28352 consensus_queue.cc:260] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:40.787955 28352 ts_tablet_manager.cc:1428] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:40.788074 28357 tablet_replica.cc:439] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } }
I20260619 08:23:40.788206 28357 tablet_replica.cc:442] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:23:40.803845 28359 raft_consensus.cc:482] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:23:40.803931 28359 raft_consensus.cc:504] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:40.804320 28359 leader_election.cc:288] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d5c92e30c474e38baf39c67d696d4c5 (127.25.127.129:35327), d85bad83c1ba4bcab1d6055ee0acf2e6 (127.25.127.130:37517)
I20260619 08:23:40.808218 28350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.804543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:36707 (local address 127.25.127.130:37517)
0619 08:23:40.804595 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.804597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.804608 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:23:40.804697 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.804698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.804706 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.804787 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:40.804986 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.805734 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.807233 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.807813 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.807894 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.807903 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.808086 (+ 183us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.808128 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 08:23:40.808218 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.804583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:50565 (local address 127.25.127.129:35327)
0619 08:23:40.804631 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.804633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.804767 (+ 134us) server_negotiation.cc:407] Connection header received
0619 08:23:40.804803 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.804803 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.804812 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.804855 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:40.804995 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.805753 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.807234 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.807791 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.807872 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.807883 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.808088 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.808129 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:23:40.808238 28221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.804496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:37517 (local address 127.25.127.131:36707)
0619 08:23:40.804558 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:40.804571 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:40.804625 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:40.804798 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:40.804799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:40.804801 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:40.804875 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.804877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.805825 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.805826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.807137 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.807139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.807911 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.807912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.807999 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.808003 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:23:40.808061 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 08:23:40.808114 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":73}
I20260619 08:23:40.808250 28372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.804508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:35327 (local address 127.25.127.131:50565)
0619 08:23:40.804683 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:40.804692 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:40.804741 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:40.804865 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:40.804866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:40.804867 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:40.804920 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.804922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.805825 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.805826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.807137 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.807139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.807918 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.807919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.807999 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.808003 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:23:40.808061 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 08:23:40.808114 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":67,"thread_start_us":97,"threads_started":1}
I20260619 08:23:40.808560 28002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1db91b021f744370b62ebba8c157c829" candidate_uuid: "8e267a01bb5645abb97cad8408953ed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d5c92e30c474e38baf39c67d696d4c5" is_pre_election: true
I20260619 08:23:40.808665 28002 raft_consensus.cc:2454] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e267a01bb5645abb97cad8408953ed3 in term 0.
I20260619 08:23:40.808753 28142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1db91b021f744370b62ebba8c157c829" candidate_uuid: "8e267a01bb5645abb97cad8408953ed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" is_pre_election: true
I20260619 08:23:40.808862 28142 raft_consensus.cc:2454] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e267a01bb5645abb97cad8408953ed3 in term 0.
I20260619 08:23:40.808851 28213 leader_election.cc:302] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d5c92e30c474e38baf39c67d696d4c5, 8e267a01bb5645abb97cad8408953ed3; no voters:
I20260619 08:23:40.808977 28359 raft_consensus.cc:2781] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:23:40.809026 28359 raft_consensus.cc:482] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:23:40.809049 28359 raft_consensus.cc:3037] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:40.809716 28359 raft_consensus.cc:504] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:40.809847 28359 leader_election.cc:288] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [CANDIDATE]: Term 1 election: Requested vote from peers 7d5c92e30c474e38baf39c67d696d4c5 (127.25.127.129:35327), d85bad83c1ba4bcab1d6055ee0acf2e6 (127.25.127.130:37517)
I20260619 08:23:40.810009 28142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1db91b021f744370b62ebba8c157c829" candidate_uuid: "8e267a01bb5645abb97cad8408953ed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6"
I20260619 08:23:40.810024 28002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1db91b021f744370b62ebba8c157c829" candidate_uuid: "8e267a01bb5645abb97cad8408953ed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d5c92e30c474e38baf39c67d696d4c5"
I20260619 08:23:40.810107 28142 raft_consensus.cc:3037] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:40.810107 28002 raft_consensus.cc:3037] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:40.810756 28002 raft_consensus.cc:2454] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e267a01bb5645abb97cad8408953ed3 in term 1.
I20260619 08:23:40.810869 28142 raft_consensus.cc:2454] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e267a01bb5645abb97cad8408953ed3 in term 1.
I20260619 08:23:40.810920 28213 leader_election.cc:302] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d5c92e30c474e38baf39c67d696d4c5, 8e267a01bb5645abb97cad8408953ed3; no voters:
I20260619 08:23:40.811030 28359 raft_consensus.cc:2781] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:40.811199 28359 raft_consensus.cc:686] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [term 1 LEADER]: Becoming Leader. State: Replica: 8e267a01bb5645abb97cad8408953ed3, State: Running, Role: LEADER
I20260619 08:23:40.811292 28359 consensus_queue.cc:237] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:40.811712 28373 tablet_replica.cc:439] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3: TxnStatusTablet state changed. Reason: New leader 8e267a01bb5645abb97cad8408953ed3. Latest consensus state: current_term: 1 leader_uuid: "8e267a01bb5645abb97cad8408953ed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } }
I20260619 08:23:40.811810 28373 tablet_replica.cc:442] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:23:40.812062 28375 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 08:23:40.812186 28375 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 08:23:40.812083 27844 catalog_manager.cc:5495] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e267a01bb5645abb97cad8408953ed3 (127.25.127.131). New cstate: current_term: 1 leader_uuid: "8e267a01bb5645abb97cad8408953ed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } health_report { overall_health: HEALTHY } } }
I20260619 08:23:40.817411 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.815012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35446 (local address 127.25.127.131:45725)
0619 08:23:40.815057 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.815059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.815068 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:23:40.815156 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.815157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.815174 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.815236 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:23:40.815355 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.815814 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.816629 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.816832 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.816903 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.817082 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:23:40.817188 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:23:40.817275 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.817318 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.817340 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:23:40.817443 28368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.814949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:45725 (local address 127.0.0.1:35446)
0619 08:23:40.815018 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:40.815030 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:40.815079 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:40.815240 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:40.815241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:40.815242 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:23:40.815295 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.815297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.815899 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.815900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.816536 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.816538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.816904 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.816904 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.816978 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.816991 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:23:40.817254 (+ 263us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:23:40.817264 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 08:23:40.817312 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 08:23:40.817362 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 08:23:40.819198 28142 raft_consensus.cc:1261] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 FOLLOWER]: Refusing update from remote peer 8e267a01bb5645abb97cad8408953ed3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:40.819516 28002 raft_consensus.cc:1261] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5 [term 1 FOLLOWER]: Refusing update from remote peer 8e267a01bb5645abb97cad8408953ed3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:40.819489 28373 consensus_queue.cc:1035] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:40.819952 28373 consensus_queue.cc:1035] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:40.820955 28358 tablet_replica.cc:439] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6: TxnStatusTablet state changed. Reason: New leader 8e267a01bb5645abb97cad8408953ed3. Latest consensus state: current_term: 1 leader_uuid: "8e267a01bb5645abb97cad8408953ed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } }
I20260619 08:23:40.821053 28358 tablet_replica.cc:442] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:23:40.821641 28359 tablet_replica.cc:439] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8e267a01bb5645abb97cad8408953ed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } }
I20260619 08:23:40.821722 28359 tablet_replica.cc:442] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:23:40.821923 28358 tablet_replica.cc:439] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8e267a01bb5645abb97cad8408953ed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } }
I20260619 08:23:40.822006 28358 tablet_replica.cc:442] T 1db91b021f744370b62ebba8c157c829 P d85bad83c1ba4bcab1d6055ee0acf2e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:23:40.822062 28357 tablet_replica.cc:439] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5: TxnStatusTablet state changed. Reason: New leader 8e267a01bb5645abb97cad8408953ed3. Latest consensus state: current_term: 1 leader_uuid: "8e267a01bb5645abb97cad8408953ed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } }
I20260619 08:23:40.822150 28357 tablet_replica.cc:442] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:23:40.822201 28378 mvcc.cc:205] Tried to move back new op lower bound from 7298487995672956928 to 7298487995643674624. Current Snapshot: MvccSnapshot[applied={T|T < 7298487995672956928}]
I20260619 08:23:40.822296 28379 mvcc.cc:205] Tried to move back new op lower bound from 7298487995672956928 to 7298487995643674624. Current Snapshot: MvccSnapshot[applied={T|T < 7298487995672956928}]
I20260619 08:23:40.823560 28373 tablet_replica.cc:439] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8e267a01bb5645abb97cad8408953ed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } }
I20260619 08:23:40.823644 28373 tablet_replica.cc:442] T 1db91b021f744370b62ebba8c157c829 P 8e267a01bb5645abb97cad8408953ed3: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:23:40.823926 28357 tablet_replica.cc:439] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8e267a01bb5645abb97cad8408953ed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } } }
I20260619 08:23:40.824009 28357 tablet_replica.cc:442] T 1db91b021f744370b62ebba8c157c829 P 7d5c92e30c474e38baf39c67d696d4c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:23:40.827422 28350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.824890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41188 (local address 127.25.127.130:37517)
0619 08:23:40.824944 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.824946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.824953 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:23:40.824985 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.824986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.825001 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.825066 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:23:40.825199 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.825941 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.826648 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.826952 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.827022 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.827134 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:23:40.827245 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:23:40.827318 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.827335 (+ 17us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.827355 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 08:23:40.828559 28383 mvcc.cc:205] Tried to move back new op lower bound from 7298487995672956928 to 7298487995643674624. Current Snapshot: MvccSnapshot[applied={T|T < 7298487995672956928}]
W20260619 08:23:40.831765 28049 tablet.cc:2369] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:23:40.831902 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.829673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:49229 (local address 127.25.127.131:45725)
0619 08:23:40.829717 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.829719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.829729 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:23:40.829773 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.829774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.829784 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.829837 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:23:40.829966 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.830419 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.831173 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.831387 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.831460 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.831635 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:23:40.831670 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:23:40.831723 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.831823 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.831842 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:23:40.831923 28194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.829601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:45725 (local address 127.25.127.130:49229)
0619 08:23:40.829669 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:40.829679 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:40.829719 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:40.829840 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:40.829841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:40.829842 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:23:40.829902 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.829903 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.830500 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.830501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.831090 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.831091 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.831459 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.831460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.831528 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.831546 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:23:40.831751 (+ 205us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:23:40.831754 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:23:40.831809 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 08:23:40.831854 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 08:23:40.834348 28002 raft_consensus.cc:1261] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5 [term 1 FOLLOWER]: Refusing update from remote peer d85bad83c1ba4bcab1d6055ee0acf2e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:40.834383 28282 raft_consensus.cc:1261] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3 [term 1 FOLLOWER]: Refusing update from remote peer d85bad83c1ba4bcab1d6055ee0acf2e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:40.834688 28358 consensus_queue.cc:1035] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:40.834915 28358 consensus_queue.cc:1035] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:40.842950 28350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.840604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:53655 (local address 127.25.127.130:37517)
0619 08:23:40.840648 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.840650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.840709 (+ 59us) server_negotiation.cc:407] Connection header received
0619 08:23:40.840778 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.840779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.840785 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.840851 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:23:40.840995 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.841442 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.842184 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.842472 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.842550 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.842717 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:23:40.842755 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:23:40.842814 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.842879 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.842905 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:23:40.842993 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.840548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:37517 (local address 127.25.127.131:53655)
0619 08:23:40.840665 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:40.840676 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:40.840724 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:40.840856 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:40.840857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:40.840859 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:23:40.840920 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.840921 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.841520 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.841521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.842105 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.842106 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.842549 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.842550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.842615 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.842632 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:23:40.842831 (+ 199us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:23:40.842834 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:23:40.842879 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 08:23:40.842923 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260619 08:23:40.849985 27844 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:23:40.850087 27844 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:23:40.850858 27844 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7: Sending DeleteTablet for 3 replicas of tablet 79c80d8e51dd4e0f8cef64535d482cf2
I20260619 08:23:40.851233 27973 tablet_service.cc:1514] Processing DeleteTablet for tablet 79c80d8e51dd4e0f8cef64535d482cf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:40 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:50742
I20260619 08:23:40.851471 28396 tablet_replica.cc:330] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5: stopping tablet replica
I20260619 08:23:40.851465 28122 tablet_service.cc:1514] Processing DeleteTablet for tablet 79c80d8e51dd4e0f8cef64535d482cf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:40 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:41186
I20260619 08:23:40.851642 28396 raft_consensus.cc:2229] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:23:40.851692 28262 tablet_service.cc:1514] Processing DeleteTablet for tablet 79c80d8e51dd4e0f8cef64535d482cf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:40 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:35434
I20260619 08:23:40.851743 28396 raft_consensus.cc:2258] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:40.851876 27844 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:23:40.852012 27844 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:23:40.852406 28397 tablet_replica.cc:330] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6: stopping tablet replica
I20260619 08:23:40.852512 28397 raft_consensus.cc:2229] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:23:40.852596 28397 raft_consensus.cc:2258] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:40.852725 28398 tablet_replica.cc:330] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3: stopping tablet replica
I20260619 08:23:40.852804 28398 raft_consensus.cc:2229] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:23:40.852852 28398 raft_consensus.cc:2258] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:40.853230 28398 ts_tablet_manager.cc:1905] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:40.853629 27973 tablet_service.cc:1467] Processing CreateTablet for tablet f4f580f994f14979b7949a60d6848c86 (DEFAULT_TABLE table=test-table [id=7b9c34f60e2f498cb2fbcacb5ad3b760]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:40.853691 28122 tablet_service.cc:1467] Processing CreateTablet for tablet f4f580f994f14979b7949a60d6848c86 (DEFAULT_TABLE table=test-table [id=7b9c34f60e2f498cb2fbcacb5ad3b760]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:40.853741 27973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4f580f994f14979b7949a60d6848c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:40.853796 28122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4f580f994f14979b7949a60d6848c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:40.855124 28397 ts_tablet_manager.cc:1905] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:40.855131 28262 tablet_service.cc:1467] Processing CreateTablet for tablet f4f580f994f14979b7949a60d6848c86 (DEFAULT_TABLE table=test-table [id=7b9c34f60e2f498cb2fbcacb5ad3b760]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:40.855233 28352 tablet_bootstrap.cc:492] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5: Bootstrap starting.
I20260619 08:23:40.855250 28353 tablet_bootstrap.cc:492] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6: Bootstrap starting.
I20260619 08:23:40.855630 28352 tablet_bootstrap.cc:654] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:40.855644 28353 tablet_bootstrap.cc:654] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:40.856062 28262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4f580f994f14979b7949a60d6848c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:40.856721 28396 ts_tablet_manager.cc:1905] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:40.857414 28351 tablet_bootstrap.cc:492] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3: Bootstrap starting.
I20260619 08:23:40.857977 28351 tablet_bootstrap.cc:654] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:40.858741 28396 ts_tablet_manager.cc:1918] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:23:40.858856 28396 log.cc:1196] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/wal/wals/79c80d8e51dd4e0f8cef64535d482cf2
I20260619 08:23:40.859200 28396 ts_tablet_manager.cc:1939] T 79c80d8e51dd4e0f8cef64535d482cf2 P 7d5c92e30c474e38baf39c67d696d4c5: Deleting consensus metadata
I20260619 08:23:40.859649 27829 catalog_manager.cc:4841] TS 7d5c92e30c474e38baf39c67d696d4c5 (127.25.127.129:35327): tablet 79c80d8e51dd4e0f8cef64535d482cf2 (table test-table [id=7c79cc6293ad4f51866b351b2d2d1086]) successfully deleted
I20260619 08:23:40.859723 28398 ts_tablet_manager.cc:1918] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:23:40.859794 28398 log.cc:1196] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/79c80d8e51dd4e0f8cef64535d482cf2
I20260619 08:23:40.860040 28398 ts_tablet_manager.cc:1939] T 79c80d8e51dd4e0f8cef64535d482cf2 P 8e267a01bb5645abb97cad8408953ed3: Deleting consensus metadata
I20260619 08:23:40.860219 28352 tablet_bootstrap.cc:492] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5: No bootstrap required, opened a new log
I20260619 08:23:40.860240 28397 ts_tablet_manager.cc:1918] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:23:40.860260 28352 ts_tablet_manager.cc:1397] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 08:23:40.860301 28397 log.cc:1196] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/79c80d8e51dd4e0f8cef64535d482cf2
I20260619 08:23:40.860409 28352 raft_consensus.cc:348] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:40.860476 28352 raft_consensus.cc:374] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:40.860497 28352 raft_consensus.cc:729] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d5c92e30c474e38baf39c67d696d4c5, State: Initialized, Role: FOLLOWER
I20260619 08:23:40.860544 28352 consensus_queue.cc:260] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:40.860594 28397 ts_tablet_manager.cc:1939] T 79c80d8e51dd4e0f8cef64535d482cf2 P d85bad83c1ba4bcab1d6055ee0acf2e6: Deleting consensus metadata
I20260619 08:23:40.860633 28352 ts_tablet_manager.cc:1428] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:40.860937 27830 catalog_manager.cc:4841] TS d85bad83c1ba4bcab1d6055ee0acf2e6 (127.25.127.130:37517): tablet 79c80d8e51dd4e0f8cef64535d482cf2 (table test-table [id=7c79cc6293ad4f51866b351b2d2d1086]) successfully deleted
I20260619 08:23:40.861307 27830 catalog_manager.cc:4841] TS 8e267a01bb5645abb97cad8408953ed3 (127.25.127.131:45725): tablet 79c80d8e51dd4e0f8cef64535d482cf2 (table test-table [id=7c79cc6293ad4f51866b351b2d2d1086]) successfully deleted
I20260619 08:23:40.861680 28353 tablet_bootstrap.cc:492] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6: No bootstrap required, opened a new log
I20260619 08:23:40.861729 28353 ts_tablet_manager.cc:1397] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260619 08:23:40.861862 28353 raft_consensus.cc:348] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:40.861932 28353 raft_consensus.cc:374] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:40.861953 28353 raft_consensus.cc:729] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d85bad83c1ba4bcab1d6055ee0acf2e6, State: Initialized, Role: FOLLOWER
I20260619 08:23:40.861997 28353 consensus_queue.cc:260] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:40.862080 28353 ts_tablet_manager.cc:1428] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:40.864585 28351 tablet_bootstrap.cc:492] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3: No bootstrap required, opened a new log
I20260619 08:23:40.864634 28351 ts_tablet_manager.cc:1397] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260619 08:23:40.864789 28351 raft_consensus.cc:348] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:40.864868 28351 raft_consensus.cc:374] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:40.864900 28351 raft_consensus.cc:729] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e267a01bb5645abb97cad8408953ed3, State: Initialized, Role: FOLLOWER
I20260619 08:23:40.864943 28351 consensus_queue.cc:260] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:40.865051 28351 ts_tablet_manager.cc:1428] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:40.909279 28359 raft_consensus.cc:482] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:23:40.909399 28359 raft_consensus.cc:504] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:40.909559 28359 leader_election.cc:288] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d5c92e30c474e38baf39c67d696d4c5 (127.25.127.129:35327), d85bad83c1ba4bcab1d6055ee0acf2e6 (127.25.127.130:37517)
I20260619 08:23:40.909797 28002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4f580f994f14979b7949a60d6848c86" candidate_uuid: "8e267a01bb5645abb97cad8408953ed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d5c92e30c474e38baf39c67d696d4c5" is_pre_election: true
I20260619 08:23:40.909788 28142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4f580f994f14979b7949a60d6848c86" candidate_uuid: "8e267a01bb5645abb97cad8408953ed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" is_pre_election: true
I20260619 08:23:40.909904 28002 raft_consensus.cc:2454] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e267a01bb5645abb97cad8408953ed3 in term 0.
I20260619 08:23:40.909905 28142 raft_consensus.cc:2454] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e267a01bb5645abb97cad8408953ed3 in term 0.
I20260619 08:23:40.910056 28213 leader_election.cc:302] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d5c92e30c474e38baf39c67d696d4c5, 8e267a01bb5645abb97cad8408953ed3; no voters:
I20260619 08:23:40.910151 28359 raft_consensus.cc:2781] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:23:40.910180 28359 raft_consensus.cc:482] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:23:40.910198 28359 raft_consensus.cc:3037] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:40.911125 28359 raft_consensus.cc:504] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:40.911259 28359 leader_election.cc:288] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [CANDIDATE]: Term 1 election: Requested vote from peers 7d5c92e30c474e38baf39c67d696d4c5 (127.25.127.129:35327), d85bad83c1ba4bcab1d6055ee0acf2e6 (127.25.127.130:37517)
I20260619 08:23:40.911437 28142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4f580f994f14979b7949a60d6848c86" candidate_uuid: "8e267a01bb5645abb97cad8408953ed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6"
I20260619 08:23:40.911465 28002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4f580f994f14979b7949a60d6848c86" candidate_uuid: "8e267a01bb5645abb97cad8408953ed3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d5c92e30c474e38baf39c67d696d4c5"
I20260619 08:23:40.911526 28142 raft_consensus.cc:3037] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:40.911530 28002 raft_consensus.cc:3037] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:40.912307 28002 raft_consensus.cc:2454] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e267a01bb5645abb97cad8408953ed3 in term 1.
I20260619 08:23:40.912328 28142 raft_consensus.cc:2454] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e267a01bb5645abb97cad8408953ed3 in term 1.
I20260619 08:23:40.912472 28213 leader_election.cc:302] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d5c92e30c474e38baf39c67d696d4c5, 8e267a01bb5645abb97cad8408953ed3; no voters:
I20260619 08:23:40.912557 28359 raft_consensus.cc:2781] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:40.912597 28359 raft_consensus.cc:686] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [term 1 LEADER]: Becoming Leader. State: Replica: 8e267a01bb5645abb97cad8408953ed3, State: Running, Role: LEADER
I20260619 08:23:40.912639 28359 consensus_queue.cc:237] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:40.913190 27844 catalog_manager.cc:5495] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e267a01bb5645abb97cad8408953ed3 (127.25.127.131). New cstate: current_term: 1 leader_uuid: "8e267a01bb5645abb97cad8408953ed3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } health_report { overall_health: HEALTHY } } }
W20260619 08:23:40.918031 28329 tablet.cc:2369] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:23:40.936050 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.932974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35456 (local address 127.25.127.131:45725)
0619 08:23:40.933016 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.933019 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.933027 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:23:40.933052 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.933053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.933063 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.933120 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:23:40.933262 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.933994 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.935048 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.935343 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.935423 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.935648 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:23:40.935686 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:23:40.935767 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.935944 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.935966 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:23:40.941941 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.938873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:59067 (local address 127.25.127.131:45725)
0619 08:23:40.939002 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:40.939004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:40.939009 (+ 5us) server_negotiation.cc:407] Connection header received
0619 08:23:40.939028 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:40.939028 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:40.939037 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:40.939087 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:23:40.939253 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.939995 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.941093 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.941371 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.941445 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.941611 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:23:40.941649 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:23:40.941702 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:40.941775 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 08:23:40.941802 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":113}
I20260619 08:23:40.942016 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:40.938781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:45725 (local address 127.25.127.131:59067)
0619 08:23:40.938849 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:40.938869 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:40.938925 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:40.939121 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:40.939122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:40.939124 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:23:40.939171 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.939173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.940098 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.940099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.940991 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:40.940994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:40.941462 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:40.941463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:40.941524 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:40.941530 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:23:40.941718 (+ 188us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:23:40.941721 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:23:40.941773 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:23:40.941833 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":3}
I20260619 08:23:40.945036 28002 raft_consensus.cc:1261] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5 [term 1 FOLLOWER]: Refusing update from remote peer 8e267a01bb5645abb97cad8408953ed3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:40.945024 28142 raft_consensus.cc:1261] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 FOLLOWER]: Refusing update from remote peer 8e267a01bb5645abb97cad8408953ed3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:40.945255 28385 consensus_queue.cc:1035] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:40.945510 28385 consensus_queue.cc:1035] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 08:23:40.951424 28378 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 08:23:42.474702 27844 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:23:42.474838 27844 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:23:42.475777 27844 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7: Sending DeleteTablet for 3 replicas of tablet f4f580f994f14979b7949a60d6848c86
I20260619 08:23:42.476058 28122 tablet_service.cc:1514] Processing DeleteTablet for tablet f4f580f994f14979b7949a60d6848c86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:42 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:41186
I20260619 08:23:42.476070 27973 tablet_service.cc:1514] Processing DeleteTablet for tablet f4f580f994f14979b7949a60d6848c86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:42 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:50742
I20260619 08:23:42.476096 28261 tablet_service.cc:1514] Processing DeleteTablet for tablet f4f580f994f14979b7949a60d6848c86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:42 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:35434
I20260619 08:23:42.476329 28427 tablet_replica.cc:330] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3: stopping tablet replica
I20260619 08:23:42.476392 28427 raft_consensus.cc:2229] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:23:42.476390 28426 tablet_replica.cc:330] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5: stopping tablet replica
I20260619 08:23:42.476451 28426 raft_consensus.cc:2229] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:23:42.476480 28427 raft_consensus.cc:2258] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:42.476491 28426 raft_consensus.cc:2258] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:42.476743 28426 ts_tablet_manager.cc:1905] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:42.477540 28427 ts_tablet_manager.cc:1905] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:42.478750 28426 ts_tablet_manager.cc:1918] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 08:23:42.478897 28426 log.cc:1196] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/wal/wals/f4f580f994f14979b7949a60d6848c86
I20260619 08:23:42.479067 28429 tablet_replica.cc:330] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6: stopping tablet replica
I20260619 08:23:42.479146 28429 raft_consensus.cc:2229] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:23:42.479226 28426 ts_tablet_manager.cc:1939] T f4f580f994f14979b7949a60d6848c86 P 7d5c92e30c474e38baf39c67d696d4c5: Deleting consensus metadata
I20260619 08:23:42.479245 28429 raft_consensus.cc:2258] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:42.479388 28427 ts_tablet_manager.cc:1918] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 08:23:42.479460 28427 log.cc:1196] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/f4f580f994f14979b7949a60d6848c86
I20260619 08:23:42.479525 28429 ts_tablet_manager.cc:1905] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:42.479624 27829 catalog_manager.cc:4841] TS 7d5c92e30c474e38baf39c67d696d4c5 (127.25.127.129:35327): tablet f4f580f994f14979b7949a60d6848c86 (table test-table [id=7b9c34f60e2f498cb2fbcacb5ad3b760]) successfully deleted
I20260619 08:23:42.479723 28427 ts_tablet_manager.cc:1939] T f4f580f994f14979b7949a60d6848c86 P 8e267a01bb5645abb97cad8408953ed3: Deleting consensus metadata
I20260619 08:23:42.480086 27830 catalog_manager.cc:4841] TS 8e267a01bb5645abb97cad8408953ed3 (127.25.127.131:45725): tablet f4f580f994f14979b7949a60d6848c86 (table test-table [id=7b9c34f60e2f498cb2fbcacb5ad3b760]) successfully deleted
I20260619 08:23:42.481531 28429 ts_tablet_manager.cc:1918] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 08:23:42.481601 28429 log.cc:1196] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/f4f580f994f14979b7949a60d6848c86
I20260619 08:23:42.481832 28429 ts_tablet_manager.cc:1939] T f4f580f994f14979b7949a60d6848c86 P d85bad83c1ba4bcab1d6055ee0acf2e6: Deleting consensus metadata
I20260619 08:23:42.482163 27830 catalog_manager.cc:4841] TS d85bad83c1ba4bcab1d6055ee0acf2e6 (127.25.127.130:37517): tablet f4f580f994f14979b7949a60d6848c86 (table test-table [id=7b9c34f60e2f498cb2fbcacb5ad3b760]) successfully deleted
Jun 19 08:23:42 dist-test-slave-5rhg krb5kdc[27800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857422, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 08:23:42 dist-test-slave-5rhg krb5kdc[27800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857422, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:42.503278 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:42.494997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42042 (local address 127.25.127.190:39957)
0619 08:23:42.495151 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:42.495153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:42.495181 (+ 28us) server_negotiation.cc:407] Connection header received
0619 08:23:42.495251 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:42.495252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:42.495268 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:42.495356 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:42.495935 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:42.496529 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:42.497272 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:42.497486 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:42.497563 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:42.500262 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:42.500263 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:42.500264 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:42.500272 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:42.502047 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:42.502440 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:42.502442 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:42.502442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:42.502505 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:42.502713 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:42.502714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:42.502715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:42.502843 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:42.502952 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:42.503072 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 08:23:42.503162 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260619 08:23:42.505025 27844 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:23:42.505230 27844 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:23:42.506712 28122 tablet_service.cc:1467] Processing CreateTablet for tablet b3defdc572f5413ebbd354c6714dcf89 (DEFAULT_TABLE table=test-table [id=fe5057d9a9be48c49b54cd97109f4646]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:42.506711 28261 tablet_service.cc:1467] Processing CreateTablet for tablet b3defdc572f5413ebbd354c6714dcf89 (DEFAULT_TABLE table=test-table [id=fe5057d9a9be48c49b54cd97109f4646]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:42.506691 27973 tablet_service.cc:1467] Processing CreateTablet for tablet b3defdc572f5413ebbd354c6714dcf89 (DEFAULT_TABLE table=test-table [id=fe5057d9a9be48c49b54cd97109f4646]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:42.506868 27973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3defdc572f5413ebbd354c6714dcf89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:42.506855 28261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3defdc572f5413ebbd354c6714dcf89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:42.506873 28122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3defdc572f5413ebbd354c6714dcf89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:42.508324 28437 tablet_bootstrap.cc:492] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5: Bootstrap starting.
I20260619 08:23:42.508591 28438 tablet_bootstrap.cc:492] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6: Bootstrap starting.
I20260619 08:23:42.508754 28437 tablet_bootstrap.cc:654] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:42.509037 28438 tablet_bootstrap.cc:654] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:42.509367 28439 tablet_bootstrap.cc:492] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3: Bootstrap starting.
I20260619 08:23:42.509486 28437 tablet_bootstrap.cc:492] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5: No bootstrap required, opened a new log
I20260619 08:23:42.509545 28437 ts_tablet_manager.cc:1397] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:42.509745 28439 tablet_bootstrap.cc:654] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:42.509716 28437 raft_consensus.cc:348] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:42.509791 28437 raft_consensus.cc:374] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:42.509815 28437 raft_consensus.cc:729] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d5c92e30c474e38baf39c67d696d4c5, State: Initialized, Role: FOLLOWER
I20260619 08:23:42.509866 28437 consensus_queue.cc:260] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:42.510093 28437 ts_tablet_manager.cc:1428] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:42.510171 28438 tablet_bootstrap.cc:492] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6: No bootstrap required, opened a new log
I20260619 08:23:42.510251 28438 ts_tablet_manager.cc:1397] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:23:42.510377 28438 raft_consensus.cc:348] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:42.510447 28438 raft_consensus.cc:374] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:42.510468 28438 raft_consensus.cc:729] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d85bad83c1ba4bcab1d6055ee0acf2e6, State: Initialized, Role: FOLLOWER
I20260619 08:23:42.510527 28438 consensus_queue.cc:260] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:42.510740 28438 ts_tablet_manager.cc:1428] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:42.511003 28439 tablet_bootstrap.cc:492] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3: No bootstrap required, opened a new log
I20260619 08:23:42.511050 28439 ts_tablet_manager.cc:1397] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:23:42.511168 28439 raft_consensus.cc:348] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:42.511253 28439 raft_consensus.cc:374] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:42.511289 28439 raft_consensus.cc:729] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e267a01bb5645abb97cad8408953ed3, State: Initialized, Role: FOLLOWER
I20260619 08:23:42.511335 28439 consensus_queue.cc:260] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:42.511433 28439 ts_tablet_manager.cc:1428] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:42.648216 28444 raft_consensus.cc:482] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:23:42.648336 28444 raft_consensus.cc:504] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:42.648590 28444 leader_election.cc:288] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d5c92e30c474e38baf39c67d696d4c5 (127.25.127.129:35327), 8e267a01bb5645abb97cad8408953ed3 (127.25.127.131:45725)
I20260619 08:23:42.648831 28002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3defdc572f5413ebbd354c6714dcf89" candidate_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d5c92e30c474e38baf39c67d696d4c5" is_pre_election: true
I20260619 08:23:42.648967 28002 raft_consensus.cc:2454] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d85bad83c1ba4bcab1d6055ee0acf2e6 in term 0.
I20260619 08:23:42.648939 28282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3defdc572f5413ebbd354c6714dcf89" candidate_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e267a01bb5645abb97cad8408953ed3" is_pre_election: true
I20260619 08:23:42.649142 28282 raft_consensus.cc:2454] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d85bad83c1ba4bcab1d6055ee0acf2e6 in term 0.
I20260619 08:23:42.649179 28073 leader_election.cc:302] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d5c92e30c474e38baf39c67d696d4c5, d85bad83c1ba4bcab1d6055ee0acf2e6; no voters:
I20260619 08:23:42.649291 28444 raft_consensus.cc:2781] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:23:42.649410 28444 raft_consensus.cc:482] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:23:42.649447 28444 raft_consensus.cc:3037] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:42.650172 28444 raft_consensus.cc:504] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:42.650339 28444 leader_election.cc:288] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [CANDIDATE]: Term 1 election: Requested vote from peers 7d5c92e30c474e38baf39c67d696d4c5 (127.25.127.129:35327), 8e267a01bb5645abb97cad8408953ed3 (127.25.127.131:45725)
I20260619 08:23:42.650540 28282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3defdc572f5413ebbd354c6714dcf89" candidate_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e267a01bb5645abb97cad8408953ed3"
I20260619 08:23:42.650542 28002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3defdc572f5413ebbd354c6714dcf89" candidate_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d5c92e30c474e38baf39c67d696d4c5"
I20260619 08:23:42.650635 28282 raft_consensus.cc:3037] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:42.650635 28002 raft_consensus.cc:3037] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:42.651571 28282 raft_consensus.cc:2454] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d85bad83c1ba4bcab1d6055ee0acf2e6 in term 1.
I20260619 08:23:42.651583 28002 raft_consensus.cc:2454] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d85bad83c1ba4bcab1d6055ee0acf2e6 in term 1.
I20260619 08:23:42.651765 28073 leader_election.cc:302] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d5c92e30c474e38baf39c67d696d4c5, d85bad83c1ba4bcab1d6055ee0acf2e6; no voters:
I20260619 08:23:42.651885 28444 raft_consensus.cc:2781] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:42.652081 28444 raft_consensus.cc:686] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 LEADER]: Becoming Leader. State: Replica: d85bad83c1ba4bcab1d6055ee0acf2e6, State: Running, Role: LEADER
I20260619 08:23:42.652174 28444 consensus_queue.cc:237] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:42.652861 27844 catalog_manager.cc:5495] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to d85bad83c1ba4bcab1d6055ee0acf2e6 (127.25.127.130). New cstate: current_term: 1 leader_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } health_report { overall_health: UNKNOWN } } }
I20260619 08:23:42.660003 28448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:42.657472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41192 (local address 127.25.127.130:37517)
0619 08:23:42.657643 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:42.657644 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:42.657651 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:23:42.657669 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:42.657670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:42.657679 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:42.657754 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:23:42.657921 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:42.658424 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:42.659244 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:42.659473 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:42.659547 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:42.659717 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:23:42.659753 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:23:42.659805 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:42.659862 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 08:23:42.659905 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
I20260619 08:23:42.662618 28002 raft_consensus.cc:1261] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5 [term 1 FOLLOWER]: Refusing update from remote peer d85bad83c1ba4bcab1d6055ee0acf2e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:42.662629 28282 raft_consensus.cc:1261] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3 [term 1 FOLLOWER]: Refusing update from remote peer d85bad83c1ba4bcab1d6055ee0acf2e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:42.662894 28444 consensus_queue.cc:1035] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:42.663004 28445 consensus_queue.cc:1035] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:42.673460 27845 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:23:42.673565 27845 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:23:42.674398 27845 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7: Sending DeleteTablet for 3 replicas of tablet b3defdc572f5413ebbd354c6714dcf89
I20260619 08:23:42.674695 27973 tablet_service.cc:1514] Processing DeleteTablet for tablet b3defdc572f5413ebbd354c6714dcf89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:42 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:50742
I20260619 08:23:42.674718 28122 tablet_service.cc:1514] Processing DeleteTablet for tablet b3defdc572f5413ebbd354c6714dcf89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:42 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:41186
I20260619 08:23:42.674813 28261 tablet_service.cc:1514] Processing DeleteTablet for tablet b3defdc572f5413ebbd354c6714dcf89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:42 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:35434
I20260619 08:23:42.674850 28426 tablet_replica.cc:330] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5: stopping tablet replica
I20260619 08:23:42.674913 28426 raft_consensus.cc:2229] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:23:42.674969 28426 raft_consensus.cc:2258] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:42.674958 28427 tablet_replica.cc:330] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3: stopping tablet replica
I20260619 08:23:42.675035 28427 raft_consensus.cc:2229] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:23:42.675050 28429 tablet_replica.cc:330] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6: stopping tablet replica
I20260619 08:23:42.675084 28427 raft_consensus.cc:2258] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:42.675117 28429 raft_consensus.cc:2229] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:23:42.675194 28429 raft_consensus.cc:2258] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:42.675390 28427 ts_tablet_manager.cc:1905] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:42.675312 27845 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:23:42.675482 27845 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:23:42.675550 28426 ts_tablet_manager.cc:1905] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:42.675949 28429 ts_tablet_manager.cc:1905] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:42.676870 28261 tablet_service.cc:1467] Processing CreateTablet for tablet d9b3984505f9499891865f0dec06710b (DEFAULT_TABLE table=test-table [id=c3536c60f2d740e5a0c37cdd46e72220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:42.676977 28261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9b3984505f9499891865f0dec06710b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:42.677269 28426 ts_tablet_manager.cc:1918] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:23:42.677330 28426 log.cc:1196] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-0/wal/wals/b3defdc572f5413ebbd354c6714dcf89
I20260619 08:23:42.677356 28427 ts_tablet_manager.cc:1918] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:23:42.677408 28427 log.cc:1196] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/b3defdc572f5413ebbd354c6714dcf89
I20260619 08:23:42.677567 28426 ts_tablet_manager.cc:1939] T b3defdc572f5413ebbd354c6714dcf89 P 7d5c92e30c474e38baf39c67d696d4c5: Deleting consensus metadata
I20260619 08:23:42.677623 28427 ts_tablet_manager.cc:1939] T b3defdc572f5413ebbd354c6714dcf89 P 8e267a01bb5645abb97cad8408953ed3: Deleting consensus metadata
I20260619 08:23:42.677911 28429 ts_tablet_manager.cc:1918] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:23:42.677974 28429 log.cc:1196] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/b3defdc572f5413ebbd354c6714dcf89
I20260619 08:23:42.678056 27830 catalog_manager.cc:4841] TS 8e267a01bb5645abb97cad8408953ed3 (127.25.127.131:45725): tablet b3defdc572f5413ebbd354c6714dcf89 (table test-table [id=fe5057d9a9be48c49b54cd97109f4646]) successfully deleted
I20260619 08:23:42.678193 28429 ts_tablet_manager.cc:1939] T b3defdc572f5413ebbd354c6714dcf89 P d85bad83c1ba4bcab1d6055ee0acf2e6: Deleting consensus metadata
I20260619 08:23:42.678486 28439 tablet_bootstrap.cc:492] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3: Bootstrap starting.
I20260619 08:23:42.678620 28122 tablet_service.cc:1467] Processing CreateTablet for tablet d9b3984505f9499891865f0dec06710b (DEFAULT_TABLE table=test-table [id=c3536c60f2d740e5a0c37cdd46e72220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:42.678680 27829 catalog_manager.cc:4841] TS 7d5c92e30c474e38baf39c67d696d4c5 (127.25.127.129:35327): tablet b3defdc572f5413ebbd354c6714dcf89 (table test-table [id=fe5057d9a9be48c49b54cd97109f4646]) successfully deleted
I20260619 08:23:42.678738 28122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9b3984505f9499891865f0dec06710b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:42.678853 28439 tablet_bootstrap.cc:654] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:42.678890 27973 tablet_service.cc:1467] Processing CreateTablet for tablet d9b3984505f9499891865f0dec06710b (DEFAULT_TABLE table=test-table [id=c3536c60f2d740e5a0c37cdd46e72220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:42.678977 27973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9b3984505f9499891865f0dec06710b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:42.679207 27830 catalog_manager.cc:4841] TS d85bad83c1ba4bcab1d6055ee0acf2e6 (127.25.127.130:37517): tablet b3defdc572f5413ebbd354c6714dcf89 (table test-table [id=fe5057d9a9be48c49b54cd97109f4646]) successfully deleted
I20260619 08:23:42.679557 28439 tablet_bootstrap.cc:492] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3: No bootstrap required, opened a new log
I20260619 08:23:42.679626 28439 ts_tablet_manager.cc:1397] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:42.679757 28439 raft_consensus.cc:348] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:42.679840 28439 raft_consensus.cc:374] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:42.679868 28439 raft_consensus.cc:729] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e267a01bb5645abb97cad8408953ed3, State: Initialized, Role: FOLLOWER
I20260619 08:23:42.679908 28439 consensus_queue.cc:260] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:42.680007 28439 ts_tablet_manager.cc:1428] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:42.680097 28438 tablet_bootstrap.cc:492] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6: Bootstrap starting.
I20260619 08:23:42.680222 28437 tablet_bootstrap.cc:492] T d9b3984505f9499891865f0dec06710b P 7d5c92e30c474e38baf39c67d696d4c5: Bootstrap starting.
I20260619 08:23:42.680476 28438 tablet_bootstrap.cc:654] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:42.680873 28437 tablet_bootstrap.cc:654] T d9b3984505f9499891865f0dec06710b P 7d5c92e30c474e38baf39c67d696d4c5: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:42.681080 28438 tablet_bootstrap.cc:492] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6: No bootstrap required, opened a new log
I20260619 08:23:42.681133 28438 ts_tablet_manager.cc:1397] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:42.681273 28438 raft_consensus.cc:348] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:42.681375 28438 raft_consensus.cc:374] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:42.681403 28438 raft_consensus.cc:729] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d85bad83c1ba4bcab1d6055ee0acf2e6, State: Initialized, Role: FOLLOWER
I20260619 08:23:42.681449 28438 consensus_queue.cc:260] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:42.681509 28437 tablet_bootstrap.cc:492] T d9b3984505f9499891865f0dec06710b P 7d5c92e30c474e38baf39c67d696d4c5: No bootstrap required, opened a new log
I20260619 08:23:42.681550 28438 ts_tablet_manager.cc:1428] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:42.681565 28437 ts_tablet_manager.cc:1397] T d9b3984505f9499891865f0dec06710b P 7d5c92e30c474e38baf39c67d696d4c5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:42.681713 28437 raft_consensus.cc:348] T d9b3984505f9499891865f0dec06710b P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:42.681771 28437 raft_consensus.cc:374] T d9b3984505f9499891865f0dec06710b P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:42.681787 28437 raft_consensus.cc:729] T d9b3984505f9499891865f0dec06710b P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d5c92e30c474e38baf39c67d696d4c5, State: Initialized, Role: FOLLOWER
I20260619 08:23:42.681849 28437 consensus_queue.cc:260] T d9b3984505f9499891865f0dec06710b P 7d5c92e30c474e38baf39c67d696d4c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:42.681943 28437 ts_tablet_manager.cc:1428] T d9b3984505f9499891865f0dec06710b P 7d5c92e30c474e38baf39c67d696d4c5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:42.860102 28444 raft_consensus.cc:482] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:23:42.860283 28444 raft_consensus.cc:504] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:42.860455 28444 leader_election.cc:288] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d5c92e30c474e38baf39c67d696d4c5 (127.25.127.129:35327), 8e267a01bb5645abb97cad8408953ed3 (127.25.127.131:45725)
I20260619 08:23:42.860742 28282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9b3984505f9499891865f0dec06710b" candidate_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e267a01bb5645abb97cad8408953ed3" is_pre_election: true
I20260619 08:23:42.860742 28002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9b3984505f9499891865f0dec06710b" candidate_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d5c92e30c474e38baf39c67d696d4c5" is_pre_election: true
I20260619 08:23:42.860859 28282 raft_consensus.cc:2454] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d85bad83c1ba4bcab1d6055ee0acf2e6 in term 0.
I20260619 08:23:42.860859 28002 raft_consensus.cc:2454] T d9b3984505f9499891865f0dec06710b P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d85bad83c1ba4bcab1d6055ee0acf2e6 in term 0.
I20260619 08:23:42.861042 28073 leader_election.cc:302] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d5c92e30c474e38baf39c67d696d4c5, d85bad83c1ba4bcab1d6055ee0acf2e6; no voters:
I20260619 08:23:42.861181 28444 raft_consensus.cc:2781] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:23:42.861219 28444 raft_consensus.cc:482] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:23:42.861238 28444 raft_consensus.cc:3037] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:42.861960 28444 raft_consensus.cc:504] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:42.862105 28444 leader_election.cc:288] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [CANDIDATE]: Term 1 election: Requested vote from peers 7d5c92e30c474e38baf39c67d696d4c5 (127.25.127.129:35327), 8e267a01bb5645abb97cad8408953ed3 (127.25.127.131:45725)
I20260619 08:23:42.862267 28282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9b3984505f9499891865f0dec06710b" candidate_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e267a01bb5645abb97cad8408953ed3"
I20260619 08:23:42.862267 28002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9b3984505f9499891865f0dec06710b" candidate_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d5c92e30c474e38baf39c67d696d4c5"
I20260619 08:23:42.862349 28282 raft_consensus.cc:3037] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:42.862349 28002 raft_consensus.cc:3037] T d9b3984505f9499891865f0dec06710b P 7d5c92e30c474e38baf39c67d696d4c5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:42.863135 28002 raft_consensus.cc:2454] T d9b3984505f9499891865f0dec06710b P 7d5c92e30c474e38baf39c67d696d4c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d85bad83c1ba4bcab1d6055ee0acf2e6 in term 1.
I20260619 08:23:42.863135 28282 raft_consensus.cc:2454] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d85bad83c1ba4bcab1d6055ee0acf2e6 in term 1.
I20260619 08:23:42.863310 28074 leader_election.cc:302] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e267a01bb5645abb97cad8408953ed3, d85bad83c1ba4bcab1d6055ee0acf2e6; no voters:
I20260619 08:23:42.863442 28444 raft_consensus.cc:2781] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:42.863493 28444 raft_consensus.cc:686] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 LEADER]: Becoming Leader. State: Replica: d85bad83c1ba4bcab1d6055ee0acf2e6, State: Running, Role: LEADER
I20260619 08:23:42.863538 28444 consensus_queue.cc:237] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } }
I20260619 08:23:42.864084 27845 catalog_manager.cc:5495] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to d85bad83c1ba4bcab1d6055ee0acf2e6 (127.25.127.130). New cstate: current_term: 1 leader_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d85bad83c1ba4bcab1d6055ee0acf2e6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 37517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 } health_report { overall_health: UNKNOWN } } }
I20260619 08:23:42.913154 28002 raft_consensus.cc:1261] T d9b3984505f9499891865f0dec06710b P 7d5c92e30c474e38baf39c67d696d4c5 [term 1 FOLLOWER]: Refusing update from remote peer d85bad83c1ba4bcab1d6055ee0acf2e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:42.913187 28282 raft_consensus.cc:1261] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3 [term 1 FOLLOWER]: Refusing update from remote peer d85bad83c1ba4bcab1d6055ee0acf2e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:42.913475 28445 consensus_queue.cc:1035] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e267a01bb5645abb97cad8408953ed3" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:42.913627 28444 consensus_queue.cc:1035] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d5c92e30c474e38baf39c67d696d4c5" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 35327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 08:23:42.917904 28410 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 08:23:44.436177 27845 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:23:44.436403 27845 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:23:44.437322 27845 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa50403c81e64a6a87fb6ea9d49c88e7: Sending DeleteTablet for 3 replicas of tablet d9b3984505f9499891865f0dec06710b
I20260619 08:23:44.437646 28261 tablet_service.cc:1514] Processing DeleteTablet for tablet d9b3984505f9499891865f0dec06710b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:44 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:35434
I20260619 08:23:44.437743 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 27917
I20260619 08:23:44.438231 28122 tablet_service.cc:1514] Processing DeleteTablet for tablet d9b3984505f9499891865f0dec06710b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:44 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:41186
I20260619 08:23:44.438427 28481 tablet_replica.cc:330] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6: stopping tablet replica
I20260619 08:23:44.438499 28481 raft_consensus.cc:2229] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:23:44.438597 28481 raft_consensus.cc:2258] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:44.438838 28481 ts_tablet_manager.cc:1905] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:44.440267 28481 ts_tablet_manager.cc:1918] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 08:23:44.440337 28481 log.cc:1196] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/d9b3984505f9499891865f0dec06710b
I20260619 08:23:44.440601 28481 ts_tablet_manager.cc:1939] T d9b3984505f9499891865f0dec06710b P d85bad83c1ba4bcab1d6055ee0acf2e6: Deleting consensus metadata
I20260619 08:23:44.440939 27830 catalog_manager.cc:4841] TS d85bad83c1ba4bcab1d6055ee0acf2e6 (127.25.127.130:37517): tablet d9b3984505f9499891865f0dec06710b (table test-table [id=c3536c60f2d740e5a0c37cdd46e72220]) successfully deleted
W20260619 08:23:44.444209 27829 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.127.129:35327 (error 108)
I20260619 08:23:44.444330 28480 tablet_replica.cc:330] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3: stopping tablet replica
I20260619 08:23:44.444413 28480 raft_consensus.cc:2229] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:23:44.444442 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 28057
I20260619 08:23:44.444478 28480 raft_consensus.cc:2258] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:44.444782 28480 ts_tablet_manager.cc:1905] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:44.445325 28483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:44.445066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:35327 (local address 127.0.0.1:50756)
0619 08:23:44.445215 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:44.445272 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.129:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260619 08:23:44.446858 28480 ts_tablet_manager.cc:1918] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 08:23:44.446915 28480 log.cc:1196] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/d9b3984505f9499891865f0dec06710b
I20260619 08:23:44.447144 28480 ts_tablet_manager.cc:1939] T d9b3984505f9499891865f0dec06710b P 8e267a01bb5645abb97cad8408953ed3: Deleting consensus metadata
I20260619 08:23:44.447528 27830 catalog_manager.cc:4841] TS 8e267a01bb5645abb97cad8408953ed3 (127.25.127.131:45725): tablet d9b3984505f9499891865f0dec06710b (table test-table [id=c3536c60f2d740e5a0c37cdd46e72220]) successfully deleted
W20260619 08:23:44.448287 27829 catalog_manager.cc:4584] TS 7d5c92e30c474e38baf39c67d696d4c5 (127.25.127.129:35327): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d9b3984505f9499891865f0dec06710b: Network error: Client connection negotiation failed: client connection to 127.25.127.129:35327: connect: Connection refused (error 111)
I20260619 08:23:44.450932 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 28197
I20260619 08:23:44.457072 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 27813
2026-06-19T08:23:44Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5571 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:23:44 dist-test-slave-5rhg krb5kdc[28486](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:23:44 dist-test-slave-5rhg krb5kdc[28486](info): set up 2 sockets
Jun 19 08:23:44 dist-test-slave-5rhg krb5kdc[28486](info): commencing operation
krb5kdc: starting...
W20260619 08:23:45.504818 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:23:45 dist-test-slave-5rhg krb5kdc[28486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:23:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:23:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:23:45.622898 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:44977
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:43377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:44977
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:45.704023 28499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:45.704272 28499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:45.704296 28499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:45.705516 28499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:23:45.705560 28499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:45.705574 28499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:45.705587 28499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:23:45.705600 28499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:23:45.707125 28499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:43377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:44977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:44977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:45.707410 28499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:45.707648 28499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:45.712555 28505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:45.712563 28504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:45.712631 28507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:45.712849 28499 server_base.cc:1034] running on GCE node
I20260619 08:23:45.713060 28499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:45.713312 28499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:45.714500 28499 hybrid_clock.cc:648] HybridClock initialized: now 1781857425714479 us; error 30 us; skew 500 ppm
Jun 19 08:23:45 dist-test-slave-5rhg krb5kdc[28486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857425, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:45.717154 28499 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:23:45.717648 28499 webserver.cc:460] Webserver started at http://127.25.127.190:41421/ using document root <none> and password file <none>
I20260619 08:23:45.717891 28499 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:45.717960 28499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:45.718113 28499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:45.719202 28499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "1042b194742b4f2488ec34916b3ee5d3"
format_stamp: "Formatted at 2026-06-19 08:23:45 on dist-test-slave-5rhg"
server_key: "de745529ea9fa4e41c625a946a625b9d"
server_key_iv: "76dc2733cf2e3a9fc88253e79207d46d"
server_key_version: "encryptionkey@0"
I20260619 08:23:45.719585 28499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "1042b194742b4f2488ec34916b3ee5d3"
format_stamp: "Formatted at 2026-06-19 08:23:45 on dist-test-slave-5rhg"
server_key: "de745529ea9fa4e41c625a946a625b9d"
server_key_iv: "76dc2733cf2e3a9fc88253e79207d46d"
server_key_version: "encryptionkey@0"
I20260619 08:23:45.721011 28499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:23:45.721755 28514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:45.721926 28499 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:45.722031 28499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "1042b194742b4f2488ec34916b3ee5d3"
format_stamp: "Formatted at 2026-06-19 08:23:45 on dist-test-slave-5rhg"
server_key: "de745529ea9fa4e41c625a946a625b9d"
server_key_iv: "76dc2733cf2e3a9fc88253e79207d46d"
server_key_version: "encryptionkey@0"
I20260619 08:23:45.722105 28499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:45.728639 28499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:45.730978 28499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:45.731143 28499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:45.735152 28499 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:44977
I20260619 08:23:45.735181 28566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:44977 every 8 connection(s)
I20260619 08:23:45.735540 28499 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:23:45.736481 28567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:45.737802 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 28499
I20260619 08:23:45.737911 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:23:45.738090 26110 external_mini_cluster.cc:1428] Setting key f45e7f03c0b58ece364870be404871b7
I20260619 08:23:45.739092 28567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3: Bootstrap starting.
I20260619 08:23:45.739777 28567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:45.740140 28567 log.cc:824] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:45.740926 28567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3: No bootstrap required, opened a new log
I20260619 08:23:45.743224 28567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1042b194742b4f2488ec34916b3ee5d3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44977 } }
I20260619 08:23:45.743371 28567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:45.743414 28567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1042b194742b4f2488ec34916b3ee5d3, State: Initialized, Role: FOLLOWER
I20260619 08:23:45.743554 28567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1042b194742b4f2488ec34916b3ee5d3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44977 } }
I20260619 08:23:45.743647 28567 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:23:45.743686 28567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:23:45.743731 28567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:45.744547 28567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1042b194742b4f2488ec34916b3ee5d3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44977 } }
I20260619 08:23:45.744673 28567 leader_election.cc:302] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1042b194742b4f2488ec34916b3ee5d3; no voters:
I20260619 08:23:45.744848 28567 leader_election.cc:288] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:23:45.744874 28572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:45.745046 28572 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3 [term 1 LEADER]: Becoming Leader. State: Replica: 1042b194742b4f2488ec34916b3ee5d3, State: Running, Role: LEADER
I20260619 08:23:45.745116 28567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:45.745177 28572 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1042b194742b4f2488ec34916b3ee5d3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44977 } }
Jun 19 08:23:45 dist-test-slave-5rhg krb5kdc[28486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:45.746243 28573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1042b194742b4f2488ec34916b3ee5d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1042b194742b4f2488ec34916b3ee5d3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44977 } } }
I20260619 08:23:45.746354 28573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3 [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:45.746480 28573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1042b194742b4f2488ec34916b3ee5d3. Latest consensus state: current_term: 1 leader_uuid: "1042b194742b4f2488ec34916b3ee5d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1042b194742b4f2488ec34916b3ee5d3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44977 } } }
I20260619 08:23:45.746681 28573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3 [sys.catalog]: This master's current role is: LEADER
W20260619 08:23:45.747165 28586 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:23:45.747221 28586 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:23:45.747363 28587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:23:45.747510 28587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:23:45.749130 28587 catalog_manager.cc:1347] Generated new cluster ID: a99b45a78b214543b10d86d1151c3749
I20260619 08:23:45.749191 28587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:23:45.749886 28570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:45.738642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60458 (local address 127.25.127.190:44977)
0619 08:23:45.738811 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:45.738819 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:45.738845 (+ 26us) server_negotiation.cc:407] Connection header received
0619 08:23:45.739278 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:45.739286 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:45.739335 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:45.739451 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:45.740298 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:45.741389 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:45.742449 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:45.742884 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:45.742975 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:45.746030 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:45.746034 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:45.746035 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:45.746048 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:45.748105 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:45.748539 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:45.748542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:45.748542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:45.748651 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:45.748898 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:45.748899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:45.748900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:45.749223 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:45.749356 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:45.749579 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 08:23:45.749735 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
I20260619 08:23:45.772670 28587 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:23:45.773411 28587 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:23:45.778124 28587 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1042b194742b4f2488ec34916b3ee5d3: Generated new TSK 0
I20260619 08:23:45.778316 28587 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
I20260619 08:23:45.805223 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:44977
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:43377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:45.887817 28594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:45.888067 28594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:45.888109 28594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:45.889365 28594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:45.889436 28594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:45.889478 28594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:23:45.890987 28594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:43377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:44977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:45.891265 28594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:45.891516 28594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:23:45.896731 28594 server_base.cc:1034] running on GCE node
W20260619 08:23:45.896693 28600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:45.896749 28599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:45.896700 28602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:45.897123 28594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:45.897395 28594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:45.898537 28594 hybrid_clock.cc:648] HybridClock initialized: now 1781857425898520 us; error 34 us; skew 500 ppm
Jun 19 08:23:45 dist-test-slave-5rhg krb5kdc[28486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857425, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:45.901165 28594 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:23:45.901574 28594 webserver.cc:460] Webserver started at http://127.25.127.129:39257/ using document root <none> and password file <none>
I20260619 08:23:45.901778 28594 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:45.901825 28594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:45.901916 28594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:45.902969 28594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "7cae2bb0f019494d904d12abb5c22d1d"
format_stamp: "Formatted at 2026-06-19 08:23:45 on dist-test-slave-5rhg"
server_key: "de260d14d1b5f02e062b4512f1c18e66"
server_key_iv: "710cc674b8f819246bba60e157f7cb4a"
server_key_version: "encryptionkey@0"
I20260619 08:23:45.903273 28594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "7cae2bb0f019494d904d12abb5c22d1d"
format_stamp: "Formatted at 2026-06-19 08:23:45 on dist-test-slave-5rhg"
server_key: "de260d14d1b5f02e062b4512f1c18e66"
server_key_iv: "710cc674b8f819246bba60e157f7cb4a"
server_key_version: "encryptionkey@0"
I20260619 08:23:45.904631 28594 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:23:45.905371 28609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:45.905586 28594 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:45.905692 28594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "7cae2bb0f019494d904d12abb5c22d1d"
format_stamp: "Formatted at 2026-06-19 08:23:45 on dist-test-slave-5rhg"
server_key: "de260d14d1b5f02e062b4512f1c18e66"
server_key_iv: "710cc674b8f819246bba60e157f7cb4a"
server_key_version: "encryptionkey@0"
I20260619 08:23:45.905753 28594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:45.943797 28594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:45.946458 28594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:45.946624 28594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:45.946879 28594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:45.947211 28594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:45.947245 28594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:45.947273 28594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:45.947317 28594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:45.953236 28594 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:41057
I20260619 08:23:45.953255 28722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:41057 every 8 connection(s)
I20260619 08:23:45.953646 28594 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:23:45 dist-test-slave-5rhg krb5kdc[28486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857425, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:45.961009 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 28594
I20260619 08:23:45.961114 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:23:45.961346 26110 external_mini_cluster.cc:1428] Setting key f40c273efb9fda042c016f38dbeba44c
I20260619 08:23:45.963390 28570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:45.954393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:50771 (local address 127.25.127.190:44977)
0619 08:23:45.954456 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:45.954463 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:45.954686 (+ 223us) server_negotiation.cc:407] Connection header received
0619 08:23:45.955249 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:45.955250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:45.955263 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:45.955315 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:45.956344 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:45.956858 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:45.957894 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:45.958116 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:45.958201 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:45.960450 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:45.960451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:45.960451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:45.960458 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:45.961833 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:45.962249 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:45.962250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:45.962250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:45.962309 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:45.962572 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:45.962573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:45.962573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:45.962686 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:45.962758 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:45.963189 (+ 431us) server_negotiation.cc:299] Negotiation successful
0619 08:23:45.963287 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:23:45.963618 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:45.954401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:44977 (local address 127.25.127.129:50771)
0619 08:23:45.954568 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:45.954604 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:45.955135 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:45.955390 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:45.955395 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:45.955976 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:45.956236 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:45.956242 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:45.956956 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:45.956957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:45.957792 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:45.957795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:45.958213 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:45.958214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:45.958356 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:45.958914 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:45.958922 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:45.960365 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:45.961922 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:45.961929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:45.961932 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:45.962159 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:45.962379 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:45.962380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:45.962381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:45.962502 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:45.962760 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:45.962768 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:45.963022 (+ 254us) client_negotiation.cc:769] Sending connection context
0619 08:23:45.963167 (+ 145us) client_negotiation.cc:240] Negotiation successful
0619 08:23:45.963378 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260619 08:23:45.964190 28723 heartbeater.cc:347] Connected to a master server at 127.25.127.190:44977
I20260619 08:23:45.964300 28723 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:45.964483 28723 heartbeater.cc:511] Master 127.25.127.190:44977 requested a full tablet report, sending...
I20260619 08:23:45.965054 28531 ts_manager.cc:194] Registered new tserver with Master: 7cae2bb0f019494d904d12abb5c22d1d (127.25.127.129:41057)
I20260619 08:23:45.965670 28531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:50771
WARNING: no policy specified for kudu/127.25.127.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
I20260619 08:23:45.985416 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:44977
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:43377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:46.074342 28728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:46.074569 28728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:46.074610 28728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:46.075779 28728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:46.075850 28728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:46.075899 28728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:23:46.077777 28728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:43377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:44977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:46.078027 28728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:46.078266 28728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:46.084553 28733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:46.084585 28734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:46.084561 28736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:46.084709 28728 server_base.cc:1034] running on GCE node
I20260619 08:23:46.084921 28728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:46.085188 28728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:46.086365 28728 hybrid_clock.cc:648] HybridClock initialized: now 1781857426086350 us; error 33 us; skew 500 ppm
Jun 19 08:23:46 dist-test-slave-5rhg krb5kdc[28486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857426, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:46.089676 28728 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:23:46.090166 28728 webserver.cc:460] Webserver started at http://127.25.127.130:45597/ using document root <none> and password file <none>
I20260619 08:23:46.090400 28728 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:46.090443 28728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:46.090590 28728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:46.091725 28728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "12843ae73f2e49a8aa199ceef40aff7b"
format_stamp: "Formatted at 2026-06-19 08:23:46 on dist-test-slave-5rhg"
server_key: "625695e9864c88df6021a0577514a9e8"
server_key_iv: "3d4f95c8a77453333f69a5039d8f163a"
server_key_version: "encryptionkey@0"
I20260619 08:23:46.092089 28728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "12843ae73f2e49a8aa199ceef40aff7b"
format_stamp: "Formatted at 2026-06-19 08:23:46 on dist-test-slave-5rhg"
server_key: "625695e9864c88df6021a0577514a9e8"
server_key_iv: "3d4f95c8a77453333f69a5039d8f163a"
server_key_version: "encryptionkey@0"
I20260619 08:23:46.094009 28728 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260619 08:23:46.094972 28743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:46.095201 28728 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:23:46.095304 28728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "12843ae73f2e49a8aa199ceef40aff7b"
format_stamp: "Formatted at 2026-06-19 08:23:46 on dist-test-slave-5rhg"
server_key: "625695e9864c88df6021a0577514a9e8"
server_key_iv: "3d4f95c8a77453333f69a5039d8f163a"
server_key_version: "encryptionkey@0"
I20260619 08:23:46.095379 28728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:46.117177 28728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:46.119370 28728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:46.119534 28728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:46.119776 28728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:46.120102 28728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:46.120136 28728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:46.120208 28728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:46.120265 28728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:46.126032 28728 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:36935
I20260619 08:23:46.126149 28856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:36935 every 8 connection(s)
I20260619 08:23:46.126449 28728 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:23:46.130569 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 28728
I20260619 08:23:46.130676 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:23:46.130852 26110 external_mini_cluster.cc:1428] Setting key 487cbfc3ac66a2f54a0b8a7d5f3e83c2
Jun 19 08:23:46 dist-test-slave-5rhg krb5kdc[28486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857426, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:46.136888 28570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:46.127279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:60681 (local address 127.25.127.190:44977)
0619 08:23:46.127400 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:46.127403 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:46.127593 (+ 190us) server_negotiation.cc:407] Connection header received
0619 08:23:46.128184 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:46.128186 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:46.128199 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:46.128246 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:46.129212 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:46.129718 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:46.130971 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:46.131225 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:46.131295 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:46.133712 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:46.133713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:46.133714 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:46.133722 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:46.135379 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:46.135752 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:46.135754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:46.135755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:46.135824 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:46.136126 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:46.136127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:46.136127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:46.136300 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:46.136363 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:46.136740 (+ 377us) server_negotiation.cc:299] Negotiation successful
0619 08:23:46.136824 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 08:23:46.137207 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:46.127240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:44977 (local address 127.25.127.130:60681)
0619 08:23:46.127462 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:46.127508 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:46.128041 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:46.128332 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:46.128336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:46.128911 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:46.129105 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:46.129110 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:46.129820 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:46.129821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:46.130854 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:46.130868 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:46.131319 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:46.131320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:46.131463 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:46.132046 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:46.132054 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:46.133614 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:46.135457 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:46.135463 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:46.135465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:46.135675 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:46.135913 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:46.135914 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:46.135914 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:46.136058 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:46.136402 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:46.136409 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:46.136591 (+ 182us) client_negotiation.cc:769] Sending connection context
0619 08:23:46.136722 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 08:23:46.136916 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":88,"threads_started":1}
I20260619 08:23:46.137717 28857 heartbeater.cc:347] Connected to a master server at 127.25.127.190:44977
I20260619 08:23:46.137828 28857 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:46.138016 28857 heartbeater.cc:511] Master 127.25.127.190:44977 requested a full tablet report, sending...
I20260619 08:23:46.138365 28531 ts_manager.cc:194] Registered new tserver with Master: 12843ae73f2e49a8aa199ceef40aff7b (127.25.127.130:36935)
I20260619 08:23:46.138733 28531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:60681
WARNING: no policy specified for kudu/127.25.127.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
I20260619 08:23:46.158375 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:44977
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:43377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:46.241519 28862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:46.241726 28862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:46.241751 28862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:46.242924 28862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:46.242964 28862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:46.242991 28862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:23:46.244446 28862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:43377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:44977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:46.244638 28862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:46.244865 28862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:46.249969 28867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:46.250008 28870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:46.250072 28862 server_base.cc:1034] running on GCE node
W20260619 08:23:46.250185 28868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:46.250489 28862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:46.250689 28862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:46.251842 28862 hybrid_clock.cc:648] HybridClock initialized: now 1781857426251811 us; error 51 us; skew 500 ppm
Jun 19 08:23:46 dist-test-slave-5rhg krb5kdc[28486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857426, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:46.254599 28862 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:23:46.255064 28862 webserver.cc:460] Webserver started at http://127.25.127.131:40937/ using document root <none> and password file <none>
I20260619 08:23:46.255300 28862 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:46.255375 28862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:46.255517 28862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:46.256621 28862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "5eb5541187024b35a77e3f99691f8635"
format_stamp: "Formatted at 2026-06-19 08:23:46 on dist-test-slave-5rhg"
server_key: "96a6602b11a348c2d354c67a90ca800f"
server_key_iv: "31a0dcd58899a303dcdf10a8d6fa3af1"
server_key_version: "encryptionkey@0"
I20260619 08:23:46.256981 28862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "5eb5541187024b35a77e3f99691f8635"
format_stamp: "Formatted at 2026-06-19 08:23:46 on dist-test-slave-5rhg"
server_key: "96a6602b11a348c2d354c67a90ca800f"
server_key_iv: "31a0dcd58899a303dcdf10a8d6fa3af1"
server_key_version: "encryptionkey@0"
I20260619 08:23:46.258348 28862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:23:46.259063 28877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:46.259236 28862 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:46.259354 28862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "5eb5541187024b35a77e3f99691f8635"
format_stamp: "Formatted at 2026-06-19 08:23:46 on dist-test-slave-5rhg"
server_key: "96a6602b11a348c2d354c67a90ca800f"
server_key_iv: "31a0dcd58899a303dcdf10a8d6fa3af1"
server_key_version: "encryptionkey@0"
I20260619 08:23:46.259429 28862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:46.270526 28862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:46.272998 28862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:46.273171 28862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:46.273424 28862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:46.273744 28862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:46.273804 28862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:46.273849 28862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:46.273877 28862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:46.280045 28862 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:40817
I20260619 08:23:46.280092 28990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:40817 every 8 connection(s)
I20260619 08:23:46.280464 28862 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:23:46.283378 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 28862
I20260619 08:23:46.283489 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:23:46.283653 26110 external_mini_cluster.cc:1428] Setting key bc8c4a013b8962e8f97eec50bae0aa25
Jun 19 08:23:46 dist-test-slave-5rhg krb5kdc[28486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857426, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:46.289935 28570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:46.281215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:41151 (local address 127.25.127.190:44977)
0619 08:23:46.281271 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:46.281273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:46.281451 (+ 178us) server_negotiation.cc:407] Connection header received
0619 08:23:46.281998 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:46.281999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:46.282011 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:46.282082 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:46.283024 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:46.283582 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:46.284903 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:46.285100 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:46.285215 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:46.287244 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:46.287245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:46.287246 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:46.287253 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:46.288528 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:46.288962 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:46.288964 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:46.288964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:46.289021 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:46.289282 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:46.289283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:46.289283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:46.289395 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:46.289473 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:46.289764 (+ 291us) server_negotiation.cc:299] Negotiation successful
0619 08:23:46.289841 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 08:23:46.290140 28993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:46.281178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:44977 (local address 127.25.127.131:41151)
0619 08:23:46.281316 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:46.281365 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:46.281882 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:46.282152 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:46.282156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:46.282731 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:46.282921 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:46.282926 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:46.283679 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:46.283680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:46.284819 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:46.284821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:46.285215 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:46.285216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:46.285351 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:46.285794 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:46.285802 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:46.287145 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:46.288624 (+ 1479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:46.288631 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:46.288633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:46.288872 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:46.289096 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:46.289097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:46.289097 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:46.289206 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:46.289463 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:46.289470 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:46.289615 (+ 145us) client_negotiation.cc:769] Sending connection context
0619 08:23:46.289741 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 08:23:46.289902 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260619 08:23:46.290617 28991 heartbeater.cc:347] Connected to a master server at 127.25.127.190:44977
I20260619 08:23:46.290714 28991 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:46.290858 28991 heartbeater.cc:511] Master 127.25.127.190:44977 requested a full tablet report, sending...
I20260619 08:23:46.291219 28531 ts_manager.cc:194] Registered new tserver with Master: 5eb5541187024b35a77e3f99691f8635 (127.25.127.131:40817)
I20260619 08:23:46.291605 28531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:41151
I20260619 08:23:46.295575 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 08:23:46.308472 28570 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:23:46.307270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60462 (local address 127.25.127.190:44977)
0619 08:23:46.307352 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:46.307355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:46.307436 (+ 81us) server_negotiation.cc:407] Connection header received
0619 08:23:46.307552 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:46.307554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:46.307574 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:46.307644 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:46.308379 (+ 735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60462: BlockingRecv error: recv got EOF from 127.0.0.1:60462 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 08:23:46.309556 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 28594
I20260619 08:23:46.315486 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 28728
I20260619 08:23:46.320253 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 28862
I20260619 08:23:46.324539 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 28499
2026-06-19T08:23:46Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1865 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:23:46 dist-test-slave-5rhg krb5kdc[29003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:23:46 dist-test-slave-5rhg krb5kdc[29003](info): set up 2 sockets
Jun 19 08:23:46 dist-test-slave-5rhg krb5kdc[29003](info): commencing operation
krb5kdc: starting...
W20260619 08:23:47.364879 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:23:47 dist-test-slave-5rhg krb5kdc[29003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:23:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:23:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:23:47.486400 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:42895
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:37901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:42895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:47.567137 29016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:47.567394 29016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:47.567432 29016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:47.568619 29016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:23:47.568687 29016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:47.568711 29016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:47.568732 29016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:23:47.568763 29016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:23:47.570266 29016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:37901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:42895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:42895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:47.570547 29016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:47.570775 29016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:47.575810 29022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:47.575860 29016 server_base.cc:1034] running on GCE node
W20260619 08:23:47.575819 29021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:47.575819 29024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:47.576345 29016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:47.576638 29016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:47.577792 29016 hybrid_clock.cc:648] HybridClock initialized: now 1781857427577778 us; error 29 us; skew 500 ppm
Jun 19 08:23:47 dist-test-slave-5rhg krb5kdc[29003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857427, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:47.580341 29016 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:23:47.580726 29016 webserver.cc:460] Webserver started at http://127.25.127.190:41791/ using document root <none> and password file <none>
I20260619 08:23:47.580940 29016 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:47.580981 29016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:47.581065 29016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:47.582088 29016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "2298d9738d194169ae540de08d201c38"
format_stamp: "Formatted at 2026-06-19 08:23:47 on dist-test-slave-5rhg"
server_key: "070cf1bf6011e179fb47722bdee5c122"
server_key_iv: "3195b1cd5239149f0a83af4719ec24a3"
server_key_version: "encryptionkey@0"
I20260619 08:23:47.582399 29016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "2298d9738d194169ae540de08d201c38"
format_stamp: "Formatted at 2026-06-19 08:23:47 on dist-test-slave-5rhg"
server_key: "070cf1bf6011e179fb47722bdee5c122"
server_key_iv: "3195b1cd5239149f0a83af4719ec24a3"
server_key_version: "encryptionkey@0"
I20260619 08:23:47.583644 29016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:23:47.584368 29031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:47.584535 29016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:47.584636 29016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "2298d9738d194169ae540de08d201c38"
format_stamp: "Formatted at 2026-06-19 08:23:47 on dist-test-slave-5rhg"
server_key: "070cf1bf6011e179fb47722bdee5c122"
server_key_iv: "3195b1cd5239149f0a83af4719ec24a3"
server_key_version: "encryptionkey@0"
I20260619 08:23:47.584708 29016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:47.610416 29016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:47.612526 29016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:47.612645 29016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:47.616632 29016 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:42895
I20260619 08:23:47.616667 29083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:42895 every 8 connection(s)
I20260619 08:23:47.617228 29016 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:23:47.617975 29084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:47.620380 29084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38: Bootstrap starting.
I20260619 08:23:47.620987 29084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:47.621305 29084 log.cc:824] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:47.621670 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 29016
I20260619 08:23:47.621752 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:23:47.621941 26110 external_mini_cluster.cc:1428] Setting key 2d26db954a3bcb53d16d5801f4cfeb08
I20260619 08:23:47.622130 29084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38: No bootstrap required, opened a new log
I20260619 08:23:47.625034 29084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2298d9738d194169ae540de08d201c38" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 42895 } }
I20260619 08:23:47.625208 29084 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:47.625240 29084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2298d9738d194169ae540de08d201c38, State: Initialized, Role: FOLLOWER
I20260619 08:23:47.625361 29084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2298d9738d194169ae540de08d201c38" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 42895 } }
I20260619 08:23:47.625432 29084 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:23:47.625459 29084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:23:47.625504 29084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:47.626185 29084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2298d9738d194169ae540de08d201c38" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 42895 } }
I20260619 08:23:47.626276 29084 leader_election.cc:302] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2298d9738d194169ae540de08d201c38; no voters:
I20260619 08:23:47.626459 29084 leader_election.cc:288] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:23:47.626483 29089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:47.626592 29089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38 [term 1 LEADER]: Becoming Leader. State: Replica: 2298d9738d194169ae540de08d201c38, State: Running, Role: LEADER
I20260619 08:23:47.626674 29089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2298d9738d194169ae540de08d201c38" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 42895 } }
I20260619 08:23:47.626736 29084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:47.627024 29089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2298d9738d194169ae540de08d201c38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2298d9738d194169ae540de08d201c38" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 42895 } } }
I20260619 08:23:47.627198 29089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38 [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:47.627403 29089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2298d9738d194169ae540de08d201c38. Latest consensus state: current_term: 1 leader_uuid: "2298d9738d194169ae540de08d201c38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2298d9738d194169ae540de08d201c38" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 42895 } } }
I20260619 08:23:47.627476 29089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38 [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:47.628265 29097 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:23:47.628767 29097 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 08:23:47 dist-test-slave-5rhg krb5kdc[29003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:47.630427 29097 catalog_manager.cc:1347] Generated new cluster ID: fe66babf2c69480a9558351932a223ed
I20260619 08:23:47.630477 29097 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:23:47.633812 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:47.622440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56154 (local address 127.25.127.190:42895)
0619 08:23:47.622666 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:47.622676 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:47.622712 (+ 36us) server_negotiation.cc:407] Connection header received
0619 08:23:47.623521 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:47.623531 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:47.623599 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:47.623756 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:47.624422 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:47.625844 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:47.626735 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:47.627364 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:47.627471 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:47.630427 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:47.630431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:47.630431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:47.630443 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:47.632233 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:47.632616 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:47.632619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:47.632620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:47.632725 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:47.632936 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:47.632937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:47.632938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:47.633178 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:47.633377 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:47.633454 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 08:23:47.633594 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":105,"threads_started":1}
I20260619 08:23:47.634704 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:23:47.649904 29097 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:23:47.650776 29097 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:23:47.654276 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:47.650558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56164 (local address 127.25.127.190:42895)
0619 08:23:47.650601 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:47.650603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:47.650822 (+ 219us) server_negotiation.cc:407] Connection header received
0619 08:23:47.650939 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:47.650940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:47.650951 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:47.651009 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:47.651402 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:47.651906 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:47.652743 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:47.652968 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:47.653055 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:47.653803 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:47.653804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:47.653805 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:23:47.653994 (+ 189us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:47.654011 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:23:47.654018 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:47.654088 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:47.654154 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 08:23:47.654183 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:23:47.657732 29097 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2298d9738d194169ae540de08d201c38: Generated new TSK 0
I20260619 08:23:47.657910 29097 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:23:47.660274 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 29016
2026-06-19T08:23:47Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1333 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:23:47 dist-test-slave-5rhg krb5kdc[29117](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:23:47 dist-test-slave-5rhg krb5kdc[29117](info): set up 2 sockets
Jun 19 08:23:47 dist-test-slave-5rhg krb5kdc[29117](info): commencing operation
krb5kdc: starting...
W20260619 08:23:48.696851 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:23:48 dist-test-slave-5rhg krb5kdc[29117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:23:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:23:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:23:48.815809 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:45705
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:45705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:48.895877 29130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:48.896077 29130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:48.896100 29130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:48.897274 29130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:23:48.897318 29130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:48.897333 29130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:48.897346 29130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:23:48.897358 29130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:23:48.898844 29130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:45705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:45705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:48.899089 29130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:48.899328 29130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:48.904263 29135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:48.904277 29138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:48.904412 29130 server_base.cc:1034] running on GCE node
W20260619 08:23:48.904253 29136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:48.904810 29130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:48.905109 29130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:48.906273 29130 hybrid_clock.cc:648] HybridClock initialized: now 1781857428906259 us; error 29 us; skew 500 ppm
Jun 19 08:23:48 dist-test-slave-5rhg krb5kdc[29117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857428, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:48.908845 29130 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:23:48.909320 29130 webserver.cc:460] Webserver started at http://127.25.127.190:39857/ using document root <none> and password file <none>
I20260619 08:23:48.909564 29130 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:48.909633 29130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:48.909794 29130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:48.910847 29130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "39eed7528207419aa9b604dc56f9ea5f"
format_stamp: "Formatted at 2026-06-19 08:23:48 on dist-test-slave-5rhg"
server_key: "0a5170ec8a6caca3f7ae2ed6eee0776f"
server_key_iv: "c353423183e561eb9363bc5496e0a1d5"
server_key_version: "encryptionkey@0"
I20260619 08:23:48.911238 29130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "39eed7528207419aa9b604dc56f9ea5f"
format_stamp: "Formatted at 2026-06-19 08:23:48 on dist-test-slave-5rhg"
server_key: "0a5170ec8a6caca3f7ae2ed6eee0776f"
server_key_iv: "c353423183e561eb9363bc5496e0a1d5"
server_key_version: "encryptionkey@0"
I20260619 08:23:48.912673 29130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:23:48.913414 29145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:48.913591 29130 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:48.913699 29130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "39eed7528207419aa9b604dc56f9ea5f"
format_stamp: "Formatted at 2026-06-19 08:23:48 on dist-test-slave-5rhg"
server_key: "0a5170ec8a6caca3f7ae2ed6eee0776f"
server_key_iv: "c353423183e561eb9363bc5496e0a1d5"
server_key_version: "encryptionkey@0"
I20260619 08:23:48.913776 29130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:48.927644 29130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:48.929677 29130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:48.929847 29130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:48.933950 29130 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:45705
I20260619 08:23:48.933997 29197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:45705 every 8 connection(s)
I20260619 08:23:48.934355 29130 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:23:48.935345 29198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:48.937599 29198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f: Bootstrap starting.
I20260619 08:23:48.938205 29198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:48.938483 29198 log.cc:824] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:48.939149 29198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f: No bootstrap required, opened a new log
I20260619 08:23:48.940418 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 29130
I20260619 08:23:48.940500 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:23:48.940673 26110 external_mini_cluster.cc:1428] Setting key 207b5ac6a0468689dd8404fcc4ca5d45
I20260619 08:23:48.940976 29198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39eed7528207419aa9b604dc56f9ea5f" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 45705 } }
I20260619 08:23:48.941161 29198 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:48.941285 29198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39eed7528207419aa9b604dc56f9ea5f, State: Initialized, Role: FOLLOWER
I20260619 08:23:48.941524 29198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39eed7528207419aa9b604dc56f9ea5f" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 45705 } }
I20260619 08:23:48.941612 29198 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:23:48.941634 29198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:23:48.941690 29198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:48.942797 29198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39eed7528207419aa9b604dc56f9ea5f" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 45705 } }
I20260619 08:23:48.942883 29198 leader_election.cc:302] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39eed7528207419aa9b604dc56f9ea5f; no voters:
I20260619 08:23:48.943040 29198 leader_election.cc:288] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:23:48.943081 29203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:48.943233 29203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f [term 1 LEADER]: Becoming Leader. State: Replica: 39eed7528207419aa9b604dc56f9ea5f, State: Running, Role: LEADER
I20260619 08:23:48.943289 29198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:48.943393 29203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39eed7528207419aa9b604dc56f9ea5f" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 45705 } }
I20260619 08:23:48.943791 29204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39eed7528207419aa9b604dc56f9ea5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39eed7528207419aa9b604dc56f9ea5f" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 45705 } } }
I20260619 08:23:48.943876 29204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:48.944068 29205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39eed7528207419aa9b604dc56f9ea5f. Latest consensus state: current_term: 1 leader_uuid: "39eed7528207419aa9b604dc56f9ea5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39eed7528207419aa9b604dc56f9ea5f" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 45705 } } }
I20260619 08:23:48.944327 29205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f [sys.catalog]: This master's current role is: LEADER
W20260619 08:23:48.945151 29219 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:23:48.945209 29219 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:23:48.945252 29210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:23:48.945412 29210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:23:48.947592 29210 catalog_manager.cc:1347] Generated new cluster ID: 8c60e0a951eb495fb607faefcd82be4a
I20260619 08:23:48.947644 29210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:23:48 dist-test-slave-5rhg krb5kdc[29117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:48.952005 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:48.941209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49822 (local address 127.25.127.190:45705)
0619 08:23:48.941382 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:48.941390 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:48.941415 (+ 25us) server_negotiation.cc:407] Connection header received
0619 08:23:48.941506 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:48.941512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:48.941554 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:48.941713 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:48.942523 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:48.943751 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:48.944949 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:48.945530 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:48.945660 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:48.948542 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:48.948545 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:48.948546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:48.948557 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:48.950330 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:48.950822 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:48.950825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:48.950826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:48.950958 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:48.951184 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:48.951185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:48.951185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:48.951427 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:48.951600 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:48.951706 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 08:23:48.951850 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":92,"threads_started":1}
I20260619 08:23:48.962960 29210 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:23:48.963792 29210 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:23:48.969151 29210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39eed7528207419aa9b604dc56f9ea5f: Generated new TSK 0
I20260619 08:23:48.969372 29210 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
I20260619 08:23:49.000749 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:45705
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:49.083483 29224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:49.083705 29224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:49.083738 29224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:49.084978 29224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:49.085026 29224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:49.085088 29224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:23:49.086604 29224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:45705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:49.086858 29224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:49.087109 29224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:49.092263 29232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:49.092278 29230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:49.092281 29229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:49.092334 29224 server_base.cc:1034] running on GCE node
I20260619 08:23:49.092767 29224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:49.093014 29224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:49.094184 29224 hybrid_clock.cc:648] HybridClock initialized: now 1781857429094176 us; error 37 us; skew 500 ppm
Jun 19 08:23:49 dist-test-slave-5rhg krb5kdc[29117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857429, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:49.096935 29224 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:23:49.097437 29224 webserver.cc:460] Webserver started at http://127.25.127.129:35735/ using document root <none> and password file <none>
I20260619 08:23:49.097669 29224 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:49.097746 29224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:49.097899 29224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:49.098953 29224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "85335b58f96f44d8bdb1d531cda75e80"
format_stamp: "Formatted at 2026-06-19 08:23:49 on dist-test-slave-5rhg"
server_key: "e27d841cff7bc903ebe68438952c02ad"
server_key_iv: "11c0c5bf06b0addc7c5ce429e8b02936"
server_key_version: "encryptionkey@0"
I20260619 08:23:49.099326 29224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "85335b58f96f44d8bdb1d531cda75e80"
format_stamp: "Formatted at 2026-06-19 08:23:49 on dist-test-slave-5rhg"
server_key: "e27d841cff7bc903ebe68438952c02ad"
server_key_iv: "11c0c5bf06b0addc7c5ce429e8b02936"
server_key_version: "encryptionkey@0"
I20260619 08:23:49.100804 29224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:23:49.101586 29239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:49.101784 29224 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:23:49.101902 29224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "85335b58f96f44d8bdb1d531cda75e80"
format_stamp: "Formatted at 2026-06-19 08:23:49 on dist-test-slave-5rhg"
server_key: "e27d841cff7bc903ebe68438952c02ad"
server_key_iv: "11c0c5bf06b0addc7c5ce429e8b02936"
server_key_version: "encryptionkey@0"
I20260619 08:23:49.101987 29224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:49.116366 29224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:49.118908 29224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:49.119082 29224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:49.119340 29224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:49.119666 29224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:49.119722 29224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:49.119771 29224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:49.119797 29224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:49.126197 29224 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:35831
I20260619 08:23:49.126237 29352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:35831 every 8 connection(s)
I20260619 08:23:49.126583 29224 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:23:49 dist-test-slave-5rhg krb5kdc[29117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857429, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:49.136067 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 29224
I20260619 08:23:49.136104 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:49.127354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:36623 (local address 127.25.127.190:45705)
0619 08:23:49.127423 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:49.127425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:49.127601 (+ 176us) server_negotiation.cc:407] Connection header received
0619 08:23:49.128112 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:49.128113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:49.128123 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:49.128233 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:49.129215 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:49.129703 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:49.130721 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:49.130945 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:49.131002 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:49.133266 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:49.133268 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:49.133268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:49.133277 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:49.134838 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:49.135203 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:49.135204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:49.135204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:49.135254 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:49.135461 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:49.135462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:49.135462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:49.135568 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:49.135618 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:49.135891 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 08:23:49.135998 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:23:49.136114 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:49.127306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:45705 (local address 127.25.127.129:36623)
0619 08:23:49.127466 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:49.127510 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:49.128024 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:49.128304 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:49.128308 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:49.128873 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:49.129071 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:49.129075 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:49.129812 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:49.129813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:49.130640 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:49.130642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:49.131027 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:49.131028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:49.131134 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:49.131737 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:49.131746 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:49.133158 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:49.134912 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:49.134917 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:49.134919 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:49.135136 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:49.135311 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:49.135312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:49.135312 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:49.135407 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:49.135622 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:49.135627 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:49.135740 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 08:23:49.135828 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 08:23:49.135949 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260619 08:23:49.136248 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:23:49.136449 26110 external_mini_cluster.cc:1428] Setting key c857ae36d551e329c1ccae12bf062887
I20260619 08:23:49.136790 29353 heartbeater.cc:347] Connected to a master server at 127.25.127.190:45705
I20260619 08:23:49.136901 29353 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:49.137100 29353 heartbeater.cc:511] Master 127.25.127.190:45705 requested a full tablet report, sending...
I20260619 08:23:49.137722 29162 ts_manager.cc:194] Registered new tserver with Master: 85335b58f96f44d8bdb1d531cda75e80 (127.25.127.129:35831)
I20260619 08:23:49.138614 29162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:36623
WARNING: no policy specified for kudu/127.25.127.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
I20260619 08:23:49.162832 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:45705
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:49.244432 29358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:49.244704 29358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:49.244742 29358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:49.245959 29358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:49.246026 29358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:49.246071 29358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:23:49.247524 29358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:45705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:49.247792 29358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:49.248020 29358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:49.252962 29363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:49.252995 29366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:49.253072 29358 server_base.cc:1034] running on GCE node
W20260619 08:23:49.252940 29364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:49.253502 29358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:49.253754 29358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:49.254932 29358 hybrid_clock.cc:648] HybridClock initialized: now 1781857429254918 us; error 27 us; skew 500 ppm
Jun 19 08:23:49 dist-test-slave-5rhg krb5kdc[29117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857429, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:49.257414 29358 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:23:49.257858 29358 webserver.cc:460] Webserver started at http://127.25.127.130:33087/ using document root <none> and password file <none>
I20260619 08:23:49.258080 29358 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:49.258150 29358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:49.258299 29358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:49.259327 29358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "76cd8a4bc7754cdd8bbeaad80072cddf"
format_stamp: "Formatted at 2026-06-19 08:23:49 on dist-test-slave-5rhg"
server_key: "ebef3b6d36b5afe02538187591f08d1a"
server_key_iv: "ea7519e8c70a2d0ab08174122546da3a"
server_key_version: "encryptionkey@0"
I20260619 08:23:49.259683 29358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "76cd8a4bc7754cdd8bbeaad80072cddf"
format_stamp: "Formatted at 2026-06-19 08:23:49 on dist-test-slave-5rhg"
server_key: "ebef3b6d36b5afe02538187591f08d1a"
server_key_iv: "ea7519e8c70a2d0ab08174122546da3a"
server_key_version: "encryptionkey@0"
I20260619 08:23:49.261052 29358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:23:49.261734 29373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:49.261905 29358 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:49.262002 29358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "76cd8a4bc7754cdd8bbeaad80072cddf"
format_stamp: "Formatted at 2026-06-19 08:23:49 on dist-test-slave-5rhg"
server_key: "ebef3b6d36b5afe02538187591f08d1a"
server_key_iv: "ea7519e8c70a2d0ab08174122546da3a"
server_key_version: "encryptionkey@0"
I20260619 08:23:49.262080 29358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:49.271831 29358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:49.273874 29358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:49.274041 29358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:49.274271 29358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:49.274576 29358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:49.274631 29358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:49.274673 29358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:49.274700 29358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:49.280851 29358 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:39229
I20260619 08:23:49.280894 29486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:39229 every 8 connection(s)
I20260619 08:23:49.281194 29358 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:23:49 dist-test-slave-5rhg krb5kdc[29117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857429, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:49.287899 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 29358
I20260619 08:23:49.288021 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:23:49.288269 26110 external_mini_cluster.cc:1428] Setting key c1c511471c9f85ca0f12325fbbdaa730
I20260619 08:23:49.290416 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:49.281899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:46645 (local address 127.25.127.190:45705)
0619 08:23:49.281963 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:49.281966 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:49.282183 (+ 217us) server_negotiation.cc:407] Connection header received
0619 08:23:49.282661 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:49.282662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:49.282673 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:49.282724 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:49.283633 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:49.284181 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:49.285139 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:49.285328 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:49.285379 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:49.287449 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:49.287450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:49.287450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:49.287456 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:49.289087 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:49.289461 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:49.289462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:49.289462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:49.289525 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:49.289725 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:49.289726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:49.289726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:49.289877 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:49.289943 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:49.290185 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 08:23:49.290280 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 08:23:49.290556 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:49.281880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:45705 (local address 127.25.127.130:46645)
0619 08:23:49.282064 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:49.282100 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:49.282578 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:49.282790 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:49.282794 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:49.283358 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:49.283557 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:49.283561 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:49.284264 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:49.284265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:49.285067 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:49.285068 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:49.285398 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:49.285399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:49.285499 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:49.286039 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:49.286047 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:49.287372 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:49.289162 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:49.289167 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:49.289169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:49.289386 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:49.289587 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:49.289587 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:49.289588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:49.289670 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:49.289940 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:49.289945 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:49.290060 (+ 115us) client_negotiation.cc:769] Sending connection context
0619 08:23:49.290143 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 08:23:49.290264 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
I20260619 08:23:49.291214 29487 heartbeater.cc:347] Connected to a master server at 127.25.127.190:45705
I20260619 08:23:49.291318 29487 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:49.291520 29487 heartbeater.cc:511] Master 127.25.127.190:45705 requested a full tablet report, sending...
I20260619 08:23:49.291960 29162 ts_manager.cc:194] Registered new tserver with Master: 76cd8a4bc7754cdd8bbeaad80072cddf (127.25.127.130:39229)
I20260619 08:23:49.292403 29162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:46645
WARNING: no policy specified for kudu/127.25.127.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
I20260619 08:23:49.313256 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:45705
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:49.394379 29492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:49.394582 29492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:49.394604 29492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:49.395689 29492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:49.395723 29492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:49.395748 29492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:23:49.397149 29492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:45705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:49.397404 29492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:49.397588 29492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:49.402532 29497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:49.402519 29500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:49.402535 29498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:49.402664 29492 server_base.cc:1034] running on GCE node
I20260619 08:23:49.402968 29492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:49.403228 29492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:49.404373 29492 hybrid_clock.cc:648] HybridClock initialized: now 1781857429404357 us; error 26 us; skew 500 ppm
Jun 19 08:23:49 dist-test-slave-5rhg krb5kdc[29117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857429, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:49.406904 29492 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:23:49.407402 29492 webserver.cc:460] Webserver started at http://127.25.127.131:43741/ using document root <none> and password file <none>
I20260619 08:23:49.407648 29492 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:49.407692 29492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:49.407817 29492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:49.408892 29492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "5742b9e334ac44f4b023b2894028b4f1"
format_stamp: "Formatted at 2026-06-19 08:23:49 on dist-test-slave-5rhg"
server_key: "9dc07f022d0d8ec67e7365e10648eca8"
server_key_iv: "762eb5182dc220706e237775c99155c9"
server_key_version: "encryptionkey@0"
I20260619 08:23:49.409245 29492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "5742b9e334ac44f4b023b2894028b4f1"
format_stamp: "Formatted at 2026-06-19 08:23:49 on dist-test-slave-5rhg"
server_key: "9dc07f022d0d8ec67e7365e10648eca8"
server_key_iv: "762eb5182dc220706e237775c99155c9"
server_key_version: "encryptionkey@0"
I20260619 08:23:49.410524 29492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:23:49.411231 29507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:49.411402 29492 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:23:49.411505 29492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "5742b9e334ac44f4b023b2894028b4f1"
format_stamp: "Formatted at 2026-06-19 08:23:49 on dist-test-slave-5rhg"
server_key: "9dc07f022d0d8ec67e7365e10648eca8"
server_key_iv: "762eb5182dc220706e237775c99155c9"
server_key_version: "encryptionkey@0"
I20260619 08:23:49.411577 29492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:49.422662 29492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:49.424544 29492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:49.424700 29492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:49.424922 29492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:49.425218 29492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:49.425278 29492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:49.425319 29492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:49.425346 29492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:49.431612 29492 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:43269
I20260619 08:23:49.431653 29620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:43269 every 8 connection(s)
I20260619 08:23:49.431968 29492 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:23:49.437978 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 29492
I20260619 08:23:49.438081 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
Jun 19 08:23:49 dist-test-slave-5rhg krb5kdc[29117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857429, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:49.438284 26110 external_mini_cluster.cc:1428] Setting key b7ea55280727a4ec54594fcb2c62c682
I20260619 08:23:49.441431 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:49.432709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:54493 (local address 127.25.127.190:45705)
0619 08:23:49.432779 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:49.432781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:49.432964 (+ 183us) server_negotiation.cc:407] Connection header received
0619 08:23:49.433475 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:49.433476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:49.433486 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:49.433559 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:49.434469 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:49.435039 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:49.436007 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:49.436253 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:49.436334 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:49.438835 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:49.438837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:49.438837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:49.438845 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:49.440317 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:49.440691 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:49.440692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:49.440692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:49.440744 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:49.440940 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:49.440941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:49.440941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:49.441045 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:49.441129 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:49.441306 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 08:23:49.441367 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 08:23:49.441677 29623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:49.432680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:45705 (local address 127.25.127.131:54493)
0619 08:23:49.432834 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:49.432876 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:49.433363 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:49.433614 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:49.433617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:49.434170 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:49.434377 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:49.434383 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:49.435121 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:49.435122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:49.435930 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:49.435932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:49.436346 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:49.436348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:49.436481 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:49.437029 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:49.437038 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:49.438738 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:49.440391 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:49.440398 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:49.440400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:49.440619 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:49.440800 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:49.440801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:49.440801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:49.440890 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:49.441094 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:49.441097 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:49.441206 (+ 109us) client_negotiation.cc:769] Sending connection context
0619 08:23:49.441292 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 08:23:49.441413 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260619 08:23:49.442220 29621 heartbeater.cc:347] Connected to a master server at 127.25.127.190:45705
I20260619 08:23:49.442325 29621 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:49.442502 29621 heartbeater.cc:511] Master 127.25.127.190:45705 requested a full tablet report, sending...
I20260619 08:23:49.442895 29162 ts_manager.cc:194] Registered new tserver with Master: 5742b9e334ac44f4b023b2894028b4f1 (127.25.127.131:43269)
I20260619 08:23:49.443290 29162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:54493
I20260619 08:23:49.450100 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 08:23:49 dist-test-slave-5rhg krb5kdc[29117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857429, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 08:23:49 dist-test-slave-5rhg krb5kdc[29117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857429, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:49.472517 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:49.465459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49828 (local address 127.25.127.190:45705)
0619 08:23:49.465526 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:49.465528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:49.465663 (+ 135us) server_negotiation.cc:407] Connection header received
0619 08:23:49.465712 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:49.465713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:49.465723 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:49.465791 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:49.466411 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:49.466902 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:49.467656 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:49.467844 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:49.467894 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:49.470294 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:49.470296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:49.470296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:49.470302 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:49.471496 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:49.471796 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:49.471797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:49.471797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:49.471844 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:49.472013 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:49.472014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:49.472014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:49.472207 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:49.472284 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:49.472390 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 08:23:49.472450 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260619 08:23:49.472743 29162 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49828
I20260619 08:23:49.473157 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 29224
I20260619 08:23:49.477939 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 29358
I20260619 08:23:49.482959 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 29492
I20260619 08:23:49.487288 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 29130
2026-06-19T08:23:49Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1829 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:23:49 dist-test-slave-5rhg krb5kdc[29633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:23:49 dist-test-slave-5rhg krb5kdc[29633](info): set up 2 sockets
Jun 19 08:23:49 dist-test-slave-5rhg krb5kdc[29633](info): commencing operation
krb5kdc: starting...
W20260619 08:23:50.524932 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:23:50 dist-test-slave-5rhg krb5kdc[29633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:23:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:23:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:23:50.642653 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:33353
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:33353
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:50.723518 29646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:50.723763 29646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:50.723799 29646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:50.725042 29646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:23:50.725114 29646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:50.725139 29646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:50.725168 29646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:23:50.725198 29646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:23:50.726718 29646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:33353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:33353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:50.727005 29646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:50.727242 29646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:50.732079 29654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:50.732182 29652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:50.732100 29651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:50.732414 29646 server_base.cc:1034] running on GCE node
I20260619 08:23:50.732650 29646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:50.732985 29646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:50.734175 29646 hybrid_clock.cc:648] HybridClock initialized: now 1781857430734144 us; error 43 us; skew 500 ppm
Jun 19 08:23:50 dist-test-slave-5rhg krb5kdc[29633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857430, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:50.736882 29646 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:23:50.737363 29646 webserver.cc:460] Webserver started at http://127.25.127.190:38685/ using document root <none> and password file <none>
I20260619 08:23:50.737600 29646 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:50.737668 29646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:50.737824 29646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:50.738900 29646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "88bc4add14e847359a2a7494431aa4ad"
format_stamp: "Formatted at 2026-06-19 08:23:50 on dist-test-slave-5rhg"
server_key: "546ca94d2872a2092e9f5a617fc7ebd8"
server_key_iv: "7458ae92e76a61d4fcdc32c96e4596e5"
server_key_version: "encryptionkey@0"
I20260619 08:23:50.739277 29646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "88bc4add14e847359a2a7494431aa4ad"
format_stamp: "Formatted at 2026-06-19 08:23:50 on dist-test-slave-5rhg"
server_key: "546ca94d2872a2092e9f5a617fc7ebd8"
server_key_iv: "7458ae92e76a61d4fcdc32c96e4596e5"
server_key_version: "encryptionkey@0"
I20260619 08:23:50.740681 29646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:23:50.741426 29661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:50.741595 29646 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:50.741706 29646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "88bc4add14e847359a2a7494431aa4ad"
format_stamp: "Formatted at 2026-06-19 08:23:50 on dist-test-slave-5rhg"
server_key: "546ca94d2872a2092e9f5a617fc7ebd8"
server_key_iv: "7458ae92e76a61d4fcdc32c96e4596e5"
server_key_version: "encryptionkey@0"
I20260619 08:23:50.741780 29646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:50.766233 29646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:50.768710 29646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:50.768883 29646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:50.772861 29646 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:33353
I20260619 08:23:50.772903 29713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:33353 every 8 connection(s)
I20260619 08:23:50.773432 29646 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:23:50.774083 29714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:50.776404 29714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad: Bootstrap starting.
I20260619 08:23:50.777040 29714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:50.777184 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 29646
I20260619 08:23:50.777297 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:23:50.777398 29714 log.cc:824] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:50.777483 26110 external_mini_cluster.cc:1428] Setting key 7e4683670258882304b5704b55edc1f2
I20260619 08:23:50.778211 29714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad: No bootstrap required, opened a new log
I20260619 08:23:50.780895 29714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bc4add14e847359a2a7494431aa4ad" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33353 } }
I20260619 08:23:50.781065 29714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:50.781098 29714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88bc4add14e847359a2a7494431aa4ad, State: Initialized, Role: FOLLOWER
I20260619 08:23:50.781214 29714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bc4add14e847359a2a7494431aa4ad" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33353 } }
I20260619 08:23:50.781283 29714 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:23:50.781311 29714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:23:50.781353 29714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:50.781955 29714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bc4add14e847359a2a7494431aa4ad" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33353 } }
I20260619 08:23:50.782073 29714 leader_election.cc:302] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88bc4add14e847359a2a7494431aa4ad; no voters:
I20260619 08:23:50.782238 29714 leader_election.cc:288] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:23:50.782269 29719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:50.782527 29719 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad [term 1 LEADER]: Becoming Leader. State: Replica: 88bc4add14e847359a2a7494431aa4ad, State: Running, Role: LEADER
I20260619 08:23:50.782574 29714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:50.782639 29719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bc4add14e847359a2a7494431aa4ad" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33353 } }
I20260619 08:23:50.783012 29719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88bc4add14e847359a2a7494431aa4ad. Latest consensus state: current_term: 1 leader_uuid: "88bc4add14e847359a2a7494431aa4ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bc4add14e847359a2a7494431aa4ad" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33353 } } }
I20260619 08:23:50.783166 29719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:50.783627 29728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:23:50.784183 29720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88bc4add14e847359a2a7494431aa4ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88bc4add14e847359a2a7494431aa4ad" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33353 } } }
Jun 19 08:23:50 dist-test-slave-5rhg krb5kdc[29633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:50.784451 29720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:50.784235 29728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:23:50.786345 29728 catalog_manager.cc:1347] Generated new cluster ID: 950da6e76e7241bf92506dc5d21b3ea6
I20260619 08:23:50.786398 29728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:23:50.788827 29718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:50.778028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35468 (local address 127.25.127.190:33353)
0619 08:23:50.778220 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:50.778229 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:50.778261 (+ 32us) server_negotiation.cc:407] Connection header received
0619 08:23:50.778741 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:50.778749 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:50.778797 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:50.778911 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:50.779690 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:50.780634 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:50.781674 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:50.782122 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:50.782242 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:50.785150 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:50.785154 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:50.785155 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:50.785166 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:50.787201 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:50.787583 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:50.787586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:50.787586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:50.787695 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:50.787895 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:50.787896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:50.787896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:50.788235 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:50.788390 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:50.788477 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 08:23:50.788611 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260619 08:23:50.792541 29728 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:23:50.793031 29728 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:23:50.801483 29728 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88bc4add14e847359a2a7494431aa4ad: Generated new TSK 0
I20260619 08:23:50.801677 29728 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
I20260619 08:23:50.826572 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:33353
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:50.907786 29740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:50.907998 29740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:50.908022 29740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:50.909245 29740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:50.909288 29740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:50.909317 29740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:23:50.910957 29740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:33353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:50.911201 29740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:50.911437 29740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:50.916462 29748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:50.916467 29745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:50.916467 29746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:50.916882 29740 server_base.cc:1034] running on GCE node
I20260619 08:23:50.917105 29740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:50.917407 29740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:50.918571 29740 hybrid_clock.cc:648] HybridClock initialized: now 1781857430918566 us; error 38 us; skew 500 ppm
Jun 19 08:23:50 dist-test-slave-5rhg krb5kdc[29633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857430, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:50.921123 29740 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:23:50.921587 29740 webserver.cc:460] Webserver started at http://127.25.127.129:42967/ using document root <none> and password file <none>
I20260619 08:23:50.921823 29740 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:50.921895 29740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:50.922049 29740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:50.923126 29740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "fe83e7bcb410408cb84df4d7e7fef99f"
format_stamp: "Formatted at 2026-06-19 08:23:50 on dist-test-slave-5rhg"
server_key: "2429948c2c64bdfa994785f5a21c7457"
server_key_iv: "656ea281982618d01e48c8c3d01d4a4b"
server_key_version: "encryptionkey@0"
I20260619 08:23:50.923503 29740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "fe83e7bcb410408cb84df4d7e7fef99f"
format_stamp: "Formatted at 2026-06-19 08:23:50 on dist-test-slave-5rhg"
server_key: "2429948c2c64bdfa994785f5a21c7457"
server_key_iv: "656ea281982618d01e48c8c3d01d4a4b"
server_key_version: "encryptionkey@0"
I20260619 08:23:50.924979 29740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:23:50.925745 29755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:50.925923 29740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:23:50.926029 29740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "fe83e7bcb410408cb84df4d7e7fef99f"
format_stamp: "Formatted at 2026-06-19 08:23:50 on dist-test-slave-5rhg"
server_key: "2429948c2c64bdfa994785f5a21c7457"
server_key_iv: "656ea281982618d01e48c8c3d01d4a4b"
server_key_version: "encryptionkey@0"
I20260619 08:23:50.926103 29740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:50.936017 29740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:50.938123 29740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:50.938304 29740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:50.938570 29740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:50.938894 29740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:50.938951 29740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:50.938988 29740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:50.939036 29740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:50.945034 29740 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:36689
I20260619 08:23:50.945114 29868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:36689 every 8 connection(s)
I20260619 08:23:50.945456 29740 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:23:50.951493 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 29740
I20260619 08:23:50.951584 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
Jun 19 08:23:50 dist-test-slave-5rhg krb5kdc[29633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857430, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:50.951812 26110 external_mini_cluster.cc:1428] Setting key 0e03bea6064e97d0b36dafdf88365e7d
I20260619 08:23:50.955061 29718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:50.946273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:53045 (local address 127.25.127.190:33353)
0619 08:23:50.946337 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:50.946339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:50.946532 (+ 193us) server_negotiation.cc:407] Connection header received
0619 08:23:50.947490 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:50.947491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:50.947504 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:50.947584 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:50.948279 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:50.948800 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:50.949776 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:50.949980 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:50.950034 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:50.952253 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:50.952254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:50.952255 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:50.952262 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:50.953673 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:50.954091 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:50.954093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:50.954093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:50.954154 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:50.954411 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:50.954412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:50.954413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:50.954527 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:50.954606 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:50.954920 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 08:23:50.954995 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:23:50.955336 29871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:50.946244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:33353 (local address 127.25.127.129:53045)
0619 08:23:50.946391 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:50.946434 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:50.947352 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:50.947649 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:50.947653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:50.947905 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:50.948100 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:50.948104 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:50.948882 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:50.948883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:50.949701 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:50.949703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:50.950054 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:50.950054 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:50.950152 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:50.950611 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:50.950619 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:50.952126 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:50.953765 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:50.953773 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:50.953775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:50.954001 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:50.954223 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:50.954224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:50.954224 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:50.954344 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:50.954596 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:50.954603 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:50.954779 (+ 176us) client_negotiation.cc:769] Sending connection context
0619 08:23:50.954890 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 08:23:50.955070 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
I20260619 08:23:50.955920 29869 heartbeater.cc:347] Connected to a master server at 127.25.127.190:33353
I20260619 08:23:50.956035 29869 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:50.956274 29869 heartbeater.cc:511] Master 127.25.127.190:33353 requested a full tablet report, sending...
I20260619 08:23:50.956815 29678 ts_manager.cc:194] Registered new tserver with Master: fe83e7bcb410408cb84df4d7e7fef99f (127.25.127.129:36689)
I20260619 08:23:50.957448 29678 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:53045
WARNING: no policy specified for kudu/127.25.127.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
I20260619 08:23:50.976558 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:33353
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:51.058514 29874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:51.058763 29874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:51.058801 29874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:51.059957 29874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:51.060026 29874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:51.060072 29874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:23:51.061638 29874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:33353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:51.061908 29874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:51.062153 29874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:51.067169 29880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:51.067170 29882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:51.067205 29874 server_base.cc:1034] running on GCE node
W20260619 08:23:51.067170 29879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:51.067602 29874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:51.067881 29874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:51.069060 29874 hybrid_clock.cc:648] HybridClock initialized: now 1781857431069056 us; error 36 us; skew 500 ppm
Jun 19 08:23:51 dist-test-slave-5rhg krb5kdc[29633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:51.071692 29874 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:23:51.072225 29874 webserver.cc:460] Webserver started at http://127.25.127.130:41431/ using document root <none> and password file <none>
I20260619 08:23:51.072468 29874 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:51.072541 29874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:51.072722 29874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:51.073755 29874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "b05fc9173a1248188d3dd5e6f80c2e6b"
format_stamp: "Formatted at 2026-06-19 08:23:51 on dist-test-slave-5rhg"
server_key: "2ecf1d1be740ae336dd176935e2461d2"
server_key_iv: "0af9da9f4362fe9332a08f27bfd6316f"
server_key_version: "encryptionkey@0"
I20260619 08:23:51.074116 29874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "b05fc9173a1248188d3dd5e6f80c2e6b"
format_stamp: "Formatted at 2026-06-19 08:23:51 on dist-test-slave-5rhg"
server_key: "2ecf1d1be740ae336dd176935e2461d2"
server_key_iv: "0af9da9f4362fe9332a08f27bfd6316f"
server_key_version: "encryptionkey@0"
I20260619 08:23:51.075418 29874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:23:51.076179 29889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:51.076367 29874 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:51.076463 29874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "b05fc9173a1248188d3dd5e6f80c2e6b"
format_stamp: "Formatted at 2026-06-19 08:23:51 on dist-test-slave-5rhg"
server_key: "2ecf1d1be740ae336dd176935e2461d2"
server_key_iv: "0af9da9f4362fe9332a08f27bfd6316f"
server_key_version: "encryptionkey@0"
I20260619 08:23:51.076552 29874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:51.087393 29874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:51.089902 29874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:51.090067 29874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:51.090312 29874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:51.090633 29874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:51.090689 29874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:51.090723 29874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:51.090751 29874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:51.096840 29874 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:38289
I20260619 08:23:51.096895 30002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:38289 every 8 connection(s)
I20260619 08:23:51.097221 29874 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:23:51.101435 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 29874
I20260619 08:23:51.101563 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:23:51.101735 26110 external_mini_cluster.cc:1428] Setting key 04e53731cd6a841947fb5cb9740e4bf8
Jun 19 08:23:51 dist-test-slave-5rhg krb5kdc[29633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:51.107837 30005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:51.097923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:33353 (local address 127.25.127.130:53723)
0619 08:23:51.098090 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:51.098135 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:51.099072 (+ 937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:51.099380 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:51.099384 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:51.099643 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:51.099853 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:51.099857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:51.100629 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:51.100630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:51.101463 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:51.101465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:51.101975 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:51.101976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:51.102143 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:51.102631 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:51.102642 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:51.104324 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:51.106283 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:51.106290 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:51.106292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:51.106633 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:51.106876 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:51.106877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:51.106877 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:51.107001 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:51.107323 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:51.107328 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:51.107448 (+ 120us) client_negotiation.cc:769] Sending connection context
0619 08:23:51.107548 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 08:23:51.107673 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260619 08:23:51.107839 29718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:51.097958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:53723 (local address 127.25.127.190:33353)
0619 08:23:51.098009 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:51.098011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:51.098223 (+ 212us) server_negotiation.cc:407] Connection header received
0619 08:23:51.099189 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:51.099191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:51.099203 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:51.099284 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:51.099962 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:51.100535 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:51.101579 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:51.101878 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:51.101951 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:51.104422 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:51.104424 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:51.104424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:51.104433 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:51.106192 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:51.106718 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:51.106719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:51.106719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:51.106791 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:51.107075 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:51.107076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:51.107076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:51.107228 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:51.107310 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:51.107598 (+ 288us) server_negotiation.cc:299] Negotiation successful
0619 08:23:51.107686 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:23:51.108467 30003 heartbeater.cc:347] Connected to a master server at 127.25.127.190:33353
I20260619 08:23:51.108566 30003 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:51.108711 30003 heartbeater.cc:511] Master 127.25.127.190:33353 requested a full tablet report, sending...
I20260619 08:23:51.109076 29678 ts_manager.cc:194] Registered new tserver with Master: b05fc9173a1248188d3dd5e6f80c2e6b (127.25.127.130:38289)
I20260619 08:23:51.109587 29678 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:53723
WARNING: no policy specified for kudu/127.25.127.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
I20260619 08:23:51.127272 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:33353
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:51.208566 30008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:51.208777 30008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:51.208801 30008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:51.209976 30008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:51.210017 30008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:51.210044 30008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:23:51.211555 30008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:33353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:51.211798 30008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:51.212023 30008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:51.217165 30013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:51.217131 30014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:51.217133 30016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:51.217399 30008 server_base.cc:1034] running on GCE node
I20260619 08:23:51.217656 30008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:51.217903 30008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:51.219077 30008 hybrid_clock.cc:648] HybridClock initialized: now 1781857431219063 us; error 42 us; skew 500 ppm
Jun 19 08:23:51 dist-test-slave-5rhg krb5kdc[29633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:51.221534 30008 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:23:51.222008 30008 webserver.cc:460] Webserver started at http://127.25.127.131:34337/ using document root <none> and password file <none>
I20260619 08:23:51.222245 30008 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:51.222314 30008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:51.222470 30008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:51.223489 30008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "884bd597603d46a1ad3c62b70af1a02e"
format_stamp: "Formatted at 2026-06-19 08:23:51 on dist-test-slave-5rhg"
server_key: "b403b3d6b4ca9a06217e933ff43c11eb"
server_key_iv: "9922599c3cd402ec1d174c621553dba4"
server_key_version: "encryptionkey@0"
I20260619 08:23:51.223851 30008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "884bd597603d46a1ad3c62b70af1a02e"
format_stamp: "Formatted at 2026-06-19 08:23:51 on dist-test-slave-5rhg"
server_key: "b403b3d6b4ca9a06217e933ff43c11eb"
server_key_iv: "9922599c3cd402ec1d174c621553dba4"
server_key_version: "encryptionkey@0"
I20260619 08:23:51.225426 30008 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:23:51.226116 30023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:51.226294 30008 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:23:51.226392 30008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "884bd597603d46a1ad3c62b70af1a02e"
format_stamp: "Formatted at 2026-06-19 08:23:51 on dist-test-slave-5rhg"
server_key: "b403b3d6b4ca9a06217e933ff43c11eb"
server_key_iv: "9922599c3cd402ec1d174c621553dba4"
server_key_version: "encryptionkey@0"
I20260619 08:23:51.226473 30008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:51.244877 30008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:51.246855 30008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:51.247020 30008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:51.247265 30008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:51.247574 30008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:51.247627 30008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:51.247671 30008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:51.247704 30008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:51.253592 30008 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:37347
I20260619 08:23:51.253635 30136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:37347 every 8 connection(s)
I20260619 08:23:51.253952 30008 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:23:51 dist-test-slave-5rhg krb5kdc[29633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:51.262391 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 30008
I20260619 08:23:51.262513 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:23:51.262738 26110 external_mini_cluster.cc:1428] Setting key 9e2999fc9ee0b02c0b54b915de163bc1
I20260619 08:23:51.263914 29718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:51.254704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:34357 (local address 127.25.127.190:33353)
0619 08:23:51.254771 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:51.254774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:51.254957 (+ 183us) server_negotiation.cc:407] Connection header received
0619 08:23:51.255950 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:51.255952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:51.255967 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:51.256015 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:51.256761 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:51.257285 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:51.258336 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:51.258542 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:51.258617 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:51.260988 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:51.260989 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:51.260989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:51.260996 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:51.262165 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:51.262685 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:51.262686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:51.262686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:51.262741 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:51.263097 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:51.263098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:51.263099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:51.263301 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:51.263385 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:51.263722 (+ 337us) server_negotiation.cc:299] Negotiation successful
0619 08:23:51.263823 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 08:23:51.264132 30139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:51.254675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:33353 (local address 127.25.127.131:34357)
0619 08:23:51.254836 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:51.254873 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:51.255821 (+ 948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:51.256093 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:51.256097 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:51.256429 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:51.256636 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:51.256641 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:51.257391 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:51.257392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:51.258227 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:51.258229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:51.258639 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:51.258640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:51.258772 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:51.259342 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:51.259351 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:51.260891 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:51.262264 (+ 1373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:51.262271 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:51.262274 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:51.262605 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:51.262844 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:51.262845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:51.262845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:51.263004 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:51.263406 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:51.263414 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:51.263576 (+ 162us) client_negotiation.cc:769] Sending connection context
0619 08:23:51.263698 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 08:23:51.263897 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260619 08:23:51.264762 30137 heartbeater.cc:347] Connected to a master server at 127.25.127.190:33353
I20260619 08:23:51.264845 30137 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:51.264986 30137 heartbeater.cc:511] Master 127.25.127.190:33353 requested a full tablet report, sending...
I20260619 08:23:51.265344 29678 ts_manager.cc:194] Registered new tserver with Master: 884bd597603d46a1ad3c62b70af1a02e (127.25.127.131:37347)
I20260619 08:23:51.265924 29678 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:34357
I20260619 08:23:51.274569 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 08:23:51 dist-test-slave-5rhg krb5kdc[29633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 08:23:51 dist-test-slave-5rhg krb5kdc[29633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.129@KRBTEST.COM
I20260619 08:23:51.298297 30144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:51.290403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51144 (local address 127.25.127.129:36689)
0619 08:23:51.290579 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:51.290583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:51.290593 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:23:51.290619 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:51.290621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:51.290635 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:51.290720 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:51.291310 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:51.292061 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:51.292836 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:51.293167 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:51.293252 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:51.295565 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:51.295573 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:51.295574 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:51.295580 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:51.297148 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:51.297488 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:51.297491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:51.297491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:51.297556 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:51.297777 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:51.297777 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:51.297778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:51.297905 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:51.298016 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:51.298115 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 08:23:51.298175 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":84,"threads_started":1}
Jun 19 08:23:51 dist-test-slave-5rhg krb5kdc[29633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:51.306707 29718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:51.299620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35482 (local address 127.25.127.190:33353)
0619 08:23:51.299690 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:51.299692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:51.299699 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:23:51.299764 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:51.299765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:51.299773 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:51.299808 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:51.300325 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:51.300798 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:51.301478 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:51.301699 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:51.301779 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:51.304011 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:51.304012 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:51.304013 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:51.304019 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:51.305503 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:51.305876 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:51.305877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:51.305878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:51.305937 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:51.306131 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:51.306132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:51.306133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:51.306293 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:51.306402 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:51.306515 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 08:23:51.306597 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260619 08:23:51.306977 29678 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35482
I20260619 08:23:51.307337 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 29740
I20260619 08:23:51.311857 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 29874
I20260619 08:23:51.316198 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 30008
I20260619 08:23:51.320631 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 29646
2026-06-19T08:23:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1833 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:23:51 dist-test-slave-5rhg krb5kdc[30149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:23:51 dist-test-slave-5rhg krb5kdc[30149](info): set up 2 sockets
Jun 19 08:23:51 dist-test-slave-5rhg krb5kdc[30149](info): commencing operation
krb5kdc: starting...
W20260619 08:23:52.356914 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:23:52 dist-test-slave-5rhg krb5kdc[30149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:23:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:23:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:23:52.473254 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:45133
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:45133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:52.554817 30162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:52.555012 30162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:52.555034 30162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:52.556109 30162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:23:52.556193 30162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:52.556234 30162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:52.556248 30162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:23:52.556260 30162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:23:52.557701 30162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:45133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:45133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:52.557893 30162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:52.558053 30162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:52.562883 30167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:52.562925 30168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:52.562951 30170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:52.562970 30162 server_base.cc:1034] running on GCE node
I20260619 08:23:52.563431 30162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:52.563743 30162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:52.564910 30162 hybrid_clock.cc:648] HybridClock initialized: now 1781857432564896 us; error 26 us; skew 500 ppm
Jun 19 08:23:52 dist-test-slave-5rhg krb5kdc[30149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857432, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:52.567292 30162 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:23:52.567623 30162 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:52.567719 30162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:52.567864 30162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:52.569027 30162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "6ec2e341e35d407385ad73d20d637a4f"
format_stamp: "Formatted at 2026-06-19 08:23:52 on dist-test-slave-5rhg"
server_key: "599a942e93d9eb0d4ba1a27f0c6732c0"
server_key_iv: "97584f3e44d977b2295afc01db85b37b"
server_key_version: "encryptionkey@0"
I20260619 08:23:52.569401 30162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "6ec2e341e35d407385ad73d20d637a4f"
format_stamp: "Formatted at 2026-06-19 08:23:52 on dist-test-slave-5rhg"
server_key: "599a942e93d9eb0d4ba1a27f0c6732c0"
server_key_iv: "97584f3e44d977b2295afc01db85b37b"
server_key_version: "encryptionkey@0"
I20260619 08:23:52.570657 30162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:23:52.571426 30176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:52.571593 30162 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:23:52.571705 30162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "6ec2e341e35d407385ad73d20d637a4f"
format_stamp: "Formatted at 2026-06-19 08:23:52 on dist-test-slave-5rhg"
server_key: "599a942e93d9eb0d4ba1a27f0c6732c0"
server_key_iv: "97584f3e44d977b2295afc01db85b37b"
server_key_version: "encryptionkey@0"
I20260619 08:23:52.571799 30162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:52.584316 30162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:52.586174 30162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:52.586355 30162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:52.590301 30162 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:45133
I20260619 08:23:52.590335 30228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:45133 every 8 connection(s)
I20260619 08:23:52.590646 30162 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:23:52.591673 30229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:52.594286 30229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f: Bootstrap starting.
I20260619 08:23:52.594858 30229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:52.595162 30229 log.cc:824] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:52.595791 30229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f: No bootstrap required, opened a new log
I20260619 08:23:52.597457 30229 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ec2e341e35d407385ad73d20d637a4f" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 45133 } }
I20260619 08:23:52.597594 30229 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:52.597623 30229 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ec2e341e35d407385ad73d20d637a4f, State: Initialized, Role: FOLLOWER
I20260619 08:23:52.597765 30229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ec2e341e35d407385ad73d20d637a4f" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 45133 } }
I20260619 08:23:52.597848 30229 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:23:52.597872 30229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:23:52.597899 30229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:52.598170 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 30162
I20260619 08:23:52.598277 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:23:52.598469 26110 external_mini_cluster.cc:1428] Setting key 73b0be04b9f3c127618b8855264d18ea
I20260619 08:23:52.599020 30229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ec2e341e35d407385ad73d20d637a4f" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 45133 } }
I20260619 08:23:52.599149 30229 leader_election.cc:302] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ec2e341e35d407385ad73d20d637a4f; no voters:
I20260619 08:23:52.599346 30229 leader_election.cc:288] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:23:52.599373 30234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:52.599478 30234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f [term 1 LEADER]: Becoming Leader. State: Replica: 6ec2e341e35d407385ad73d20d637a4f, State: Running, Role: LEADER
I20260619 08:23:52.599560 30234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ec2e341e35d407385ad73d20d637a4f" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 45133 } }
I20260619 08:23:52.599627 30229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:52.599949 30235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ec2e341e35d407385ad73d20d637a4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ec2e341e35d407385ad73d20d637a4f" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 45133 } } }
I20260619 08:23:52.600266 30235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:52.600015 30236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ec2e341e35d407385ad73d20d637a4f. Latest consensus state: current_term: 1 leader_uuid: "6ec2e341e35d407385ad73d20d637a4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ec2e341e35d407385ad73d20d637a4f" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 45133 } } }
I20260619 08:23:52.600524 30236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:52.600710 30238 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:23:52.601217 30238 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:23:52.602860 30238 catalog_manager.cc:1347] Generated new cluster ID: 78e6cf31fce944f088fa2de5244bcc8b
I20260619 08:23:52.602912 30238 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:23:52 dist-test-slave-5rhg krb5kdc[30149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:52.609633 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:52.599006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50194 (local address 127.25.127.190:45133)
0619 08:23:52.599263 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:52.599273 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:52.599306 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:23:52.599420 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:52.599427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:52.599484 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:52.599622 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:52.600461 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:52.601833 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:52.602932 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:52.603392 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:52.603518 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:52.605838 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:52.605842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:52.605843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:52.605854 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:52.608033 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:52.608455 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:52.608458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:52.608459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:52.608549 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:52.608765 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:52.608766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:52.608766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:52.609005 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:52.609175 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:52.609286 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 08:23:52.609427 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":208,"thread_start_us":111,"threads_started":1}
I20260619 08:23:52.625438 30238 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:23:52.626294 30238 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:23:52.631130 30238 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f: Generated new TSK 0
I20260619 08:23:52.631345 30238 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
I20260619 08:23:52.658275 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:45133
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:52.765133 30255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:52.765372 30255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:52.765401 30255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:52.766703 30255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:52.766760 30255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:52.766791 30255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:23:52.768499 30255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:45133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:52.768775 30255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:52.769024 30255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:52.774255 30260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:52.774233 30263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:52.774219 30261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:52.774572 30255 server_base.cc:1034] running on GCE node
I20260619 08:23:52.774816 30255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:52.775130 30255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:52.776295 30255 hybrid_clock.cc:648] HybridClock initialized: now 1781857432776272 us; error 41 us; skew 500 ppm
Jun 19 08:23:52 dist-test-slave-5rhg krb5kdc[30149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857432, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:52.778884 30255 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:23:52.779275 30255 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:52.779352 30255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:52.779520 30255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:52.780754 30255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "dce51ea010734d78801a2032e3ca0339"
format_stamp: "Formatted at 2026-06-19 08:23:52 on dist-test-slave-5rhg"
server_key: "d5691785b998ce39c8f419b456191df6"
server_key_iv: "27de5a343ac680bd20501130bf956e4c"
server_key_version: "encryptionkey@0"
I20260619 08:23:52.781133 30255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "dce51ea010734d78801a2032e3ca0339"
format_stamp: "Formatted at 2026-06-19 08:23:52 on dist-test-slave-5rhg"
server_key: "d5691785b998ce39c8f419b456191df6"
server_key_iv: "27de5a343ac680bd20501130bf956e4c"
server_key_version: "encryptionkey@0"
I20260619 08:23:52.782496 30255 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:23:52.783363 30269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:52.783573 30255 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:52.783684 30255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "dce51ea010734d78801a2032e3ca0339"
format_stamp: "Formatted at 2026-06-19 08:23:52 on dist-test-slave-5rhg"
server_key: "d5691785b998ce39c8f419b456191df6"
server_key_iv: "27de5a343ac680bd20501130bf956e4c"
server_key_version: "encryptionkey@0"
I20260619 08:23:52.783744 30255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:52.803887 30255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:52.806461 30255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:52.806594 30255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:52.806814 30255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:52.807189 30255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:52.807231 30255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:52.807255 30255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:52.807308 30255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:52.813333 30255 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:46707
I20260619 08:23:52.813414 30382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:46707 every 8 connection(s)
I20260619 08:23:52.813728 30255 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:23:52.814054 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 30255
I20260619 08:23:52.814256 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:23:52.814483 26110 external_mini_cluster.cc:1428] Setting key ff433daf93b2e413e2de339e7c3337dc
Jun 19 08:23:52 dist-test-slave-5rhg krb5kdc[30149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857432, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.127.130@KRBTEST.COM; defaulting to no policy
I20260619 08:23:52.825394 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:52.814559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:36693 (local address 127.25.127.190:45133)
0619 08:23:52.814612 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:52.814614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:52.814859 (+ 245us) server_negotiation.cc:407] Connection header received
0619 08:23:52.815452 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:52.815453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:52.815466 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:52.815532 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:52.816812 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:52.817319 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:52.818962 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:52.819197 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:52.819255 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:52.822131 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:52.822133 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:52.822133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:52.822144 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:52.823809 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:52.824408 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:52.824410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:52.824410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:52.824478 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:52.824748 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:52.824749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:52.824749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:52.824869 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:52.824918 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:52.825204 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 08:23:52.825322 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:23:52.825625 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:52.814553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:45133 (local address 127.25.127.129:36693)
0619 08:23:52.814740 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:52.814785 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:52.815345 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:52.815626 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:52.815630 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:52.815908 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:52.816701 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:52.816711 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:52.817421 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:52.817422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:52.818854 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:52.818858 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:52.819296 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:52.819297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:52.819478 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:52.820063 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:52.820077 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:52.821997 (+ 1920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:52.823900 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:52.823909 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:52.823911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:52.824260 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:52.824551 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:52.824551 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:52.824552 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:52.824671 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:52.824924 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:52.824928 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:52.825066 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 08:23:52.825146 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 08:23:52.825378 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":108,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.130@KRBTEST.COM" created.
I20260619 08:23:52.826166 30383 heartbeater.cc:347] Connected to a master server at 127.25.127.190:45133
I20260619 08:23:52.826300 30383 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:52.826465 30383 heartbeater.cc:511] Master 127.25.127.190:45133 requested a full tablet report, sending...
I20260619 08:23:52.827028 30193 ts_manager.cc:194] Registered new tserver with Master: dce51ea010734d78801a2032e3ca0339 (127.25.127.129:46707)
I20260619 08:23:52.828012 30193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:36693
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
I20260619 08:23:52.840526 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:45133
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:52.922457 30388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:52.922657 30388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:52.922680 30388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:52.923851 30388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:52.923895 30388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:52.923921 30388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:23:52.925442 30388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:45133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:52.925674 30388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:52.925925 30388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:23:52.931048 30388 server_base.cc:1034] running on GCE node
W20260619 08:23:52.931030 30394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:52.931031 30396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:52.931030 30393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:52.931452 30388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:52.931715 30388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:52.932884 30388 hybrid_clock.cc:648] HybridClock initialized: now 1781857432932873 us; error 45 us; skew 500 ppm
Jun 19 08:23:52 dist-test-slave-5rhg krb5kdc[30149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857432, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:52.935420 30388 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:23:52.935801 30388 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:52.935871 30388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:52.936012 30388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:52.937129 30388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "56aed3e7c81445308d5696956f6f61d6"
format_stamp: "Formatted at 2026-06-19 08:23:52 on dist-test-slave-5rhg"
server_key: "646e986f62967e15ddd91263c43ebd3c"
server_key_iv: "1c9bf416125dc50e9e173d374f133bc9"
server_key_version: "encryptionkey@0"
I20260619 08:23:52.937505 30388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "56aed3e7c81445308d5696956f6f61d6"
format_stamp: "Formatted at 2026-06-19 08:23:52 on dist-test-slave-5rhg"
server_key: "646e986f62967e15ddd91263c43ebd3c"
server_key_iv: "1c9bf416125dc50e9e173d374f133bc9"
server_key_version: "encryptionkey@0"
I20260619 08:23:52.938792 30388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:23:52.939565 30402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:52.939747 30388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:52.939855 30388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "56aed3e7c81445308d5696956f6f61d6"
format_stamp: "Formatted at 2026-06-19 08:23:52 on dist-test-slave-5rhg"
server_key: "646e986f62967e15ddd91263c43ebd3c"
server_key_iv: "1c9bf416125dc50e9e173d374f133bc9"
server_key_version: "encryptionkey@0"
I20260619 08:23:52.939942 30388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:52.951939 30388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:52.954020 30388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:52.954185 30388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:52.954423 30388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:52.954743 30388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:52.954804 30388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:52.954846 30388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:52.954872 30388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:52.960947 30388 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:42139
I20260619 08:23:52.960984 30515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:42139 every 8 connection(s)
I20260619 08:23:52.961256 30388 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:23:52.965364 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 30388
I20260619 08:23:52.965480 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:23:52.965668 26110 external_mini_cluster.cc:1428] Setting key 4e44b24548bc543ff7f33849ee149716
Jun 19 08:23:52 dist-test-slave-5rhg krb5kdc[30149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857432, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:52.971498 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:52.961968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:38651 (local address 127.25.127.190:45133)
0619 08:23:52.962029 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:52.962031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:52.962279 (+ 248us) server_negotiation.cc:407] Connection header received
0619 08:23:52.962824 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:52.962825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:52.962838 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:52.962893 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:52.963846 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:52.964431 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:52.965477 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:52.965769 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:52.965851 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:52.968388 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:52.968390 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:52.968390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:52.968398 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:52.969990 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:52.970404 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:52.970405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:52.970406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:52.970469 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:52.970751 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:52.970752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:52.970752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:52.970926 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:52.971005 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:52.971255 (+ 250us) server_negotiation.cc:299] Negotiation successful
0619 08:23:52.971352 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 08:23:52.971515 30518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:52.961941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:45133 (local address 127.25.127.130:38651)
0619 08:23:52.962115 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:52.962176 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:52.962705 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:52.962971 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:52.962974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:52.963248 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:52.963744 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:52.963750 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:52.964542 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:52.964543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:52.965370 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:52.965372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:52.965858 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:52.965859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:52.966022 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:52.966668 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:52.966677 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:52.968294 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:52.970066 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:52.970073 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:52.970074 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:52.970307 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:52.970543 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:52.970544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:52.970544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:52.970673 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:52.970997 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:52.971002 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:52.971121 (+ 119us) client_negotiation.cc:769] Sending connection context
0619 08:23:52.971205 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 08:23:52.971349 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260619 08:23:52.972256 30516 heartbeater.cc:347] Connected to a master server at 127.25.127.190:45133
I20260619 08:23:52.972388 30516 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:52.972541 30516 heartbeater.cc:511] Master 127.25.127.190:45133 requested a full tablet report, sending...
I20260619 08:23:52.972952 30193 ts_manager.cc:194] Registered new tserver with Master: 56aed3e7c81445308d5696956f6f61d6 (127.25.127.130:42139)
I20260619 08:23:52.973325 30193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:38651
WARNING: no policy specified for kudu/127.25.127.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
I20260619 08:23:52.991545 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:45133
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:53.078928 30521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:53.079188 30521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:53.079221 30521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:53.080600 30521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:53.080654 30521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:53.080687 30521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:23:53.082489 30521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:45133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:53.082787 30521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:53.083069 30521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:53.088497 30529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:53.088491 30526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:53.088480 30527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:53.088761 30521 server_base.cc:1034] running on GCE node
I20260619 08:23:53.089087 30521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:53.089354 30521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:53.090528 30521 hybrid_clock.cc:648] HybridClock initialized: now 1781857433090515 us; error 27 us; skew 500 ppm
Jun 19 08:23:53 dist-test-slave-5rhg krb5kdc[30149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857433, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:53.093166 30521 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:23:53.093514 30521 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:53.093627 30521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:53.093773 30521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:53.094858 30521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "745faba18e5d425398090711941b2ca9"
format_stamp: "Formatted at 2026-06-19 08:23:53 on dist-test-slave-5rhg"
server_key: "36bd25fa5e90dc3b9805c319fef21089"
server_key_iv: "795530400b8c72549e90eadfbfef8b97"
server_key_version: "encryptionkey@0"
I20260619 08:23:53.095232 30521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "745faba18e5d425398090711941b2ca9"
format_stamp: "Formatted at 2026-06-19 08:23:53 on dist-test-slave-5rhg"
server_key: "36bd25fa5e90dc3b9805c319fef21089"
server_key_iv: "795530400b8c72549e90eadfbfef8b97"
server_key_version: "encryptionkey@0"
I20260619 08:23:53.096634 30521 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:23:53.097427 30535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:53.097602 30521 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:23:53.097756 30521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "745faba18e5d425398090711941b2ca9"
format_stamp: "Formatted at 2026-06-19 08:23:53 on dist-test-slave-5rhg"
server_key: "36bd25fa5e90dc3b9805c319fef21089"
server_key_iv: "795530400b8c72549e90eadfbfef8b97"
server_key_version: "encryptionkey@0"
I20260619 08:23:53.097936 30521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:53.108783 30521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:53.111096 30521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:53.111263 30521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:53.111507 30521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:53.111824 30521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:53.111878 30521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:53.111923 30521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:53.111948 30521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:53.118054 30521 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:42975
I20260619 08:23:53.118088 30648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:42975 every 8 connection(s)
I20260619 08:23:53.118368 30521 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:23:53 dist-test-slave-5rhg krb5kdc[30149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857433, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:53.126551 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 30521
I20260619 08:23:53.126667 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:23:53.126871 26110 external_mini_cluster.cc:1428] Setting key 1c970fd074baf611b22fe933d4d83aa3
I20260619 08:23:53.127802 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.119015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:36971 (local address 127.25.127.190:45133)
0619 08:23:53.119058 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.119061 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.119292 (+ 231us) server_negotiation.cc:407] Connection header received
0619 08:23:53.119771 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.119771 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.119781 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.119841 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:53.120844 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.121329 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.122295 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.122472 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.122522 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:53.124842 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:53.124843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:53.124844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:53.124849 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:53.126107 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:53.126661 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:53.126663 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:53.126663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:53.126729 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:53.127023 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:53.127024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:53.127024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:53.127144 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:53.127264 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.127605 (+ 341us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.127678 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 08:23:53.128031 30651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.118990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:45133 (local address 127.25.127.131:36971)
0619 08:23:53.119154 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:53.119198 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:53.119676 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:53.119899 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:53.119902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:53.120247 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:53.120762 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:53.120768 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.121411 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.121411 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:53.122223 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:53.122225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.122544 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.122545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:53.122646 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:53.123218 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:53.123227 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:53.124750 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:53.126206 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:53.126213 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:53.126215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:53.126555 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:53.126801 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:53.126802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:53.126802 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:53.126919 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:53.127264 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:53.127271 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:53.127447 (+ 176us) client_negotiation.cc:769] Sending connection context
0619 08:23:53.127579 (+ 132us) client_negotiation.cc:240] Negotiation successful
0619 08:23:53.127755 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260619 08:23:53.128571 30649 heartbeater.cc:347] Connected to a master server at 127.25.127.190:45133
I20260619 08:23:53.128672 30649 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:53.128852 30649 heartbeater.cc:511] Master 127.25.127.190:45133 requested a full tablet report, sending...
I20260619 08:23:53.129237 30193 ts_manager.cc:194] Registered new tserver with Master: 745faba18e5d425398090711941b2ca9 (127.25.127.131:42975)
I20260619 08:23:53.129622 30193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:36971
I20260619 08:23:53.138665 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:23:53.145865 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.139941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50198 (local address 127.25.127.190:45133)
0619 08:23:53.140011 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.140013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.140117 (+ 104us) server_negotiation.cc:407] Connection header received
0619 08:23:53.140191 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.140196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.140206 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.140274 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:53.140820 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.141334 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.142033 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.142236 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.142291 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:53.143544 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:53.143544 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:53.143545 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:53.143550 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:53.144889 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:53.145203 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:53.145204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:53.145205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:53.145273 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:53.145435 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:53.145436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:53.145436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:53.145579 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:53.145651 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.145740 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.145799 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 08:23:53.147324 30193 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50198:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:23:53.148983 30193 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:23:53.155867 30661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.151313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:42975 (local address 127.0.0.1:54168)
0619 08:23:53.152233 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:53.152242 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:53.152335 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:53.152535 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:53.152535 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:53.152538 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:53.152593 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:53.152594 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.153476 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.153478 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:53.154681 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:53.154683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.155593 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.155594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:53.155671 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:53.155685 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 08:23:53.155761 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 08:23:53.155805 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":902,"mutex_wait_us":3,"thread_start_us":103,"threads_started":1}
I20260619 08:23:53.155938 30664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.151539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54168 (local address 127.25.127.131:42975)
0619 08:23:53.151701 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.151706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.152300 (+ 594us) server_negotiation.cc:407] Connection header received
0619 08:23:53.152431 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.152434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.152455 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.152541 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:53.152656 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.153373 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.154782 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.155499 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.155577 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:53.155631 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.155791 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.155834 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":101,"threads_started":1}
I20260619 08:23:53.156426 30663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.151487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57428 (local address 127.25.127.129:46707)
0619 08:23:53.151656 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.151661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.152206 (+ 545us) server_negotiation.cc:407] Connection header received
0619 08:23:53.152498 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.152501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.152518 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.152597 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:53.152681 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.153806 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.154941 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.155511 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.155604 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:53.155665 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.156279 (+ 614us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.156336 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260619 08:23:53.156399 30662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.151305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:42139 (local address 127.0.0.1:39200)
0619 08:23:53.151869 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:53.151901 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:53.152016 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:53.152383 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:53.152387 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:53.152390 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:53.152468 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:53.152470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.153917 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.153918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:53.155071 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:53.155074 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.155946 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.155947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:53.156245 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:53.156249 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:23:53.156297 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 08:23:53.156325 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":87,"thread_start_us":59,"threads_started":1}
I20260619 08:23:53.156399 30660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.151313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:46707 (local address 127.0.0.1:57428)
0619 08:23:53.152105 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:53.152130 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:53.152398 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:53.152591 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:53.152591 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:53.152593 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:53.152631 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:53.152632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.153937 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.153938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:53.154852 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:53.154855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.156089 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.156090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:53.156177 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:53.156181 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:23:53.156244 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 08:23:53.156304 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":97,"threads_started":1}
I20260619 08:23:53.157047 30665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.151717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39200 (local address 127.25.127.130:42139)
0619 08:23:53.151871 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.151874 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.152073 (+ 199us) server_negotiation.cc:407] Connection header received
0619 08:23:53.152124 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.152127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.152143 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.152248 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:53.152684 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.153806 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.155154 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.155860 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.155931 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:53.155979 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.156904 (+ 925us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.156960 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260619 08:23:53.157229 30583 tablet_service.cc:1467] Processing CreateTablet for tablet 1dd3796e733f4e429a5365e15ec55e87 (DEFAULT_TABLE table=test-table [id=1caca0c0a81e4e9d9f57a9c55155d375]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:53.157462 30317 tablet_service.cc:1467] Processing CreateTablet for tablet 1dd3796e733f4e429a5365e15ec55e87 (DEFAULT_TABLE table=test-table [id=1caca0c0a81e4e9d9f57a9c55155d375]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:53.157619 30583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dd3796e733f4e429a5365e15ec55e87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:53.157761 30317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dd3796e733f4e429a5365e15ec55e87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:53.160717 30450 tablet_service.cc:1467] Processing CreateTablet for tablet 1dd3796e733f4e429a5365e15ec55e87 (DEFAULT_TABLE table=test-table [id=1caca0c0a81e4e9d9f57a9c55155d375]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:53.161064 30450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dd3796e733f4e429a5365e15ec55e87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:53.161248 30666 tablet_bootstrap.cc:492] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339: Bootstrap starting.
I20260619 08:23:53.161811 30666 tablet_bootstrap.cc:654] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:53.162317 30666 log.cc:824] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:53.162653 30667 tablet_bootstrap.cc:492] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9: Bootstrap starting.
I20260619 08:23:53.163187 30667 tablet_bootstrap.cc:654] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:53.163682 30667 log.cc:824] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:53.163951 30669 tablet_bootstrap.cc:492] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6: Bootstrap starting.
I20260619 08:23:53.164471 30666 tablet_bootstrap.cc:492] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339: No bootstrap required, opened a new log
I20260619 08:23:53.164559 30666 ts_tablet_manager.cc:1397] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 08:23:53.164577 30669 tablet_bootstrap.cc:654] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:53.164649 30667 tablet_bootstrap.cc:492] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9: No bootstrap required, opened a new log
I20260619 08:23:53.164727 30667 ts_tablet_manager.cc:1397] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:23:53.165148 30669 log.cc:824] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:53.165904 30669 tablet_bootstrap.cc:492] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6: No bootstrap required, opened a new log
I20260619 08:23:53.165973 30669 ts_tablet_manager.cc:1397] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:23:53.166366 30666 raft_consensus.cc:348] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce51ea010734d78801a2032e3ca0339" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46707 } } peers { permanent_uuid: "56aed3e7c81445308d5696956f6f61d6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42139 } } peers { permanent_uuid: "745faba18e5d425398090711941b2ca9" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42975 } }
I20260619 08:23:53.166522 30666 raft_consensus.cc:374] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:53.166566 30666 raft_consensus.cc:729] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dce51ea010734d78801a2032e3ca0339, State: Initialized, Role: FOLLOWER
I20260619 08:23:53.166684 30666 consensus_queue.cc:260] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce51ea010734d78801a2032e3ca0339" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46707 } } peers { permanent_uuid: "56aed3e7c81445308d5696956f6f61d6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42139 } } peers { permanent_uuid: "745faba18e5d425398090711941b2ca9" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42975 } }
I20260619 08:23:53.166839 30667 raft_consensus.cc:348] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce51ea010734d78801a2032e3ca0339" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46707 } } peers { permanent_uuid: "56aed3e7c81445308d5696956f6f61d6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42139 } } peers { permanent_uuid: "745faba18e5d425398090711941b2ca9" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42975 } }
I20260619 08:23:53.166989 30667 raft_consensus.cc:374] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:53.167001 30666 ts_tablet_manager.cc:1428] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:23:53.167023 30667 raft_consensus.cc:729] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 745faba18e5d425398090711941b2ca9, State: Initialized, Role: FOLLOWER
I20260619 08:23:53.167138 30383 heartbeater.cc:503] Master 127.25.127.190:45133 was elected leader, sending a full tablet report...
I20260619 08:23:53.167155 30667 consensus_queue.cc:260] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce51ea010734d78801a2032e3ca0339" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46707 } } peers { permanent_uuid: "56aed3e7c81445308d5696956f6f61d6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42139 } } peers { permanent_uuid: "745faba18e5d425398090711941b2ca9" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42975 } }
I20260619 08:23:53.167441 30667 ts_tablet_manager.cc:1428] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:23:53.167490 30649 heartbeater.cc:503] Master 127.25.127.190:45133 was elected leader, sending a full tablet report...
I20260619 08:23:53.168412 30669 raft_consensus.cc:348] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce51ea010734d78801a2032e3ca0339" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46707 } } peers { permanent_uuid: "56aed3e7c81445308d5696956f6f61d6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42139 } } peers { permanent_uuid: "745faba18e5d425398090711941b2ca9" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42975 } }
I20260619 08:23:53.168570 30669 raft_consensus.cc:374] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:53.168614 30669 raft_consensus.cc:729] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56aed3e7c81445308d5696956f6f61d6, State: Initialized, Role: FOLLOWER
I20260619 08:23:53.168733 30669 consensus_queue.cc:260] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce51ea010734d78801a2032e3ca0339" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46707 } } peers { permanent_uuid: "56aed3e7c81445308d5696956f6f61d6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42139 } } peers { permanent_uuid: "745faba18e5d425398090711941b2ca9" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42975 } }
I20260619 08:23:53.169021 30669 ts_tablet_manager.cc:1428] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:23:53.169044 30516 heartbeater.cc:503] Master 127.25.127.190:45133 was elected leader, sending a full tablet report...
W20260619 08:23:53.211843 30517 tablet.cc:2369] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:23:53.295456 30672 raft_consensus.cc:482] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:23:53.295646 30672 raft_consensus.cc:504] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce51ea010734d78801a2032e3ca0339" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46707 } } peers { permanent_uuid: "56aed3e7c81445308d5696956f6f61d6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42139 } } peers { permanent_uuid: "745faba18e5d425398090711941b2ca9" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42975 } }
I20260619 08:23:53.296145 30672 leader_election.cc:288] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56aed3e7c81445308d5696956f6f61d6 (127.25.127.130:42139), 745faba18e5d425398090711941b2ca9 (127.25.127.131:42975)
I20260619 08:23:53.299424 30664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.296498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:48663 (local address 127.25.127.131:42975)
0619 08:23:53.296536 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.296538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.296592 (+ 54us) server_negotiation.cc:407] Connection header received
0619 08:23:53.296684 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.296685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.296697 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.296757 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:53.296955 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.297449 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.298641 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.299029 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.299088 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:53.299096 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.299294 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.299343 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 08:23:53.299458 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.296337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:42139 (local address 127.25.127.129:55855)
0619 08:23:53.296391 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:53.296405 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:53.296451 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:53.296682 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:53.296683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:53.296685 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:53.296772 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:53.296775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.297595 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.297596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:53.298482 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:53.298485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.299161 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.299162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:53.299246 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:53.299249 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:23:53.299325 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 08:23:53.299363 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":32}
I20260619 08:23:53.299463 30665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.296415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:55855 (local address 127.25.127.130:42139)
0619 08:23:53.296522 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.296525 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.296536 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:23:53.296567 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.296568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.296579 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.296657 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:53.296875 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.297440 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.298582 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.299040 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.299150 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:53.299158 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.299349 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.299395 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260619 08:23:53.299469 30675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.296385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:42975 (local address 127.25.127.129:48663)
0619 08:23:53.296501 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:53.296514 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:53.296610 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:53.296782 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:53.296783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:53.296785 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:53.296875 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:53.296877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.297625 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.297625 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:53.298504 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:53.298506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.299126 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.299127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:53.299209 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:53.299212 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:23:53.299270 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 08:23:53.299327 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":26,"thread_start_us":68,"threads_started":1}
I20260619 08:23:53.299867 30470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dd3796e733f4e429a5365e15ec55e87" candidate_uuid: "dce51ea010734d78801a2032e3ca0339" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56aed3e7c81445308d5696956f6f61d6" is_pre_election: true
I20260619 08:23:53.299903 30603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dd3796e733f4e429a5365e15ec55e87" candidate_uuid: "dce51ea010734d78801a2032e3ca0339" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "745faba18e5d425398090711941b2ca9" is_pre_election: true
I20260619 08:23:53.300062 30470 raft_consensus.cc:2454] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dce51ea010734d78801a2032e3ca0339 in term 0.
I20260619 08:23:53.300129 30603 raft_consensus.cc:2454] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dce51ea010734d78801a2032e3ca0339 in term 0.
I20260619 08:23:53.300383 30273 leader_election.cc:302] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 745faba18e5d425398090711941b2ca9, dce51ea010734d78801a2032e3ca0339; no voters:
I20260619 08:23:53.300526 30672 raft_consensus.cc:2781] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:23:53.300595 30672 raft_consensus.cc:482] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:23:53.300613 30672 raft_consensus.cc:3037] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:53.301579 30672 raft_consensus.cc:504] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce51ea010734d78801a2032e3ca0339" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46707 } } peers { permanent_uuid: "56aed3e7c81445308d5696956f6f61d6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42139 } } peers { permanent_uuid: "745faba18e5d425398090711941b2ca9" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42975 } }
I20260619 08:23:53.301744 30672 leader_election.cc:288] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [CANDIDATE]: Term 1 election: Requested vote from peers 56aed3e7c81445308d5696956f6f61d6 (127.25.127.130:42139), 745faba18e5d425398090711941b2ca9 (127.25.127.131:42975)
I20260619 08:23:53.301950 30603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dd3796e733f4e429a5365e15ec55e87" candidate_uuid: "dce51ea010734d78801a2032e3ca0339" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "745faba18e5d425398090711941b2ca9"
I20260619 08:23:53.301950 30470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dd3796e733f4e429a5365e15ec55e87" candidate_uuid: "dce51ea010734d78801a2032e3ca0339" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56aed3e7c81445308d5696956f6f61d6"
I20260619 08:23:53.302063 30603 raft_consensus.cc:3037] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:53.302151 30470 raft_consensus.cc:3037] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:53.303216 30470 raft_consensus.cc:2454] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dce51ea010734d78801a2032e3ca0339 in term 1.
I20260619 08:23:53.303216 30603 raft_consensus.cc:2454] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dce51ea010734d78801a2032e3ca0339 in term 1.
I20260619 08:23:53.303435 30272 leader_election.cc:302] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56aed3e7c81445308d5696956f6f61d6, dce51ea010734d78801a2032e3ca0339; no voters:
I20260619 08:23:53.303552 30672 raft_consensus.cc:2781] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:53.303717 30672 raft_consensus.cc:686] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [term 1 LEADER]: Becoming Leader. State: Replica: dce51ea010734d78801a2032e3ca0339, State: Running, Role: LEADER
I20260619 08:23:53.303800 30672 consensus_queue.cc:237] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce51ea010734d78801a2032e3ca0339" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46707 } } peers { permanent_uuid: "56aed3e7c81445308d5696956f6f61d6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42139 } } peers { permanent_uuid: "745faba18e5d425398090711941b2ca9" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42975 } }
I20260619 08:23:53.304839 30192 catalog_manager.cc:5495] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 reported cstate change: term changed from 0 to 1, leader changed from <none> to dce51ea010734d78801a2032e3ca0339 (127.25.127.129). New cstate: current_term: 1 leader_uuid: "dce51ea010734d78801a2032e3ca0339" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce51ea010734d78801a2032e3ca0339" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56aed3e7c81445308d5696956f6f61d6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "745faba18e5d425398090711941b2ca9" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42975 } health_report { overall_health: UNKNOWN } } }
W20260619 08:23:53.314541 30384 tablet.cc:2369] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:23:53.336642 30663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.333998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57434 (local address 127.25.127.129:46707)
0619 08:23:53.334060 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.334062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.334069 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:23:53.334108 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.334109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.334129 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.334199 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:23:53.334323 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.334791 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.335729 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:53.335922 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:53.335981 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:53.336293 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:23:53.336420 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:23:53.336505 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.336525 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.336550 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 08:23:53.338950 30470 raft_consensus.cc:1261] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6 [term 1 FOLLOWER]: Refusing update from remote peer dce51ea010734d78801a2032e3ca0339: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:53.338954 30603 raft_consensus.cc:1261] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9 [term 1 FOLLOWER]: Refusing update from remote peer dce51ea010734d78801a2032e3ca0339: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:53.339357 30672 consensus_queue.cc:1035] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [LEADER]: Connected to new peer: Peer: permanent_uuid: "745faba18e5d425398090711941b2ca9" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:53.339519 30676 consensus_queue.cc:1035] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56aed3e7c81445308d5696956f6f61d6" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:53.344825 30192 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:23:53.344950 30192 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:23:53.345768 30192 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ec2e341e35d407385ad73d20d637a4f: Sending DeleteTablet for 3 replicas of tablet 1dd3796e733f4e429a5365e15ec55e87
I20260619 08:23:53.346158 30317 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dd3796e733f4e429a5365e15ec55e87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:53 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:57428
I20260619 08:23:53.346163 30583 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dd3796e733f4e429a5365e15ec55e87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:53 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:54168
I20260619 08:23:53.346254 30450 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dd3796e733f4e429a5365e15ec55e87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:53 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:39200
I20260619 08:23:53.346395 30688 tablet_replica.cc:330] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339: stopping tablet replica
I20260619 08:23:53.346488 30688 raft_consensus.cc:2229] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:23:53.346593 30688 raft_consensus.cc:2258] T 1dd3796e733f4e429a5365e15ec55e87 P dce51ea010734d78801a2032e3ca0339 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:53.346745 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 30255
I20260619 08:23:53.347325 30690 tablet_replica.cc:330] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6: stopping tablet replica
I20260619 08:23:53.347442 30690 raft_consensus.cc:2229] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:23:53.347558 30690 raft_consensus.cc:2258] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:53.347702 30689 tablet_replica.cc:330] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9: stopping tablet replica
I20260619 08:23:53.347801 30689 raft_consensus.cc:2229] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:23:53.347898 30689 raft_consensus.cc:2258] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:53.348484 30689 ts_tablet_manager.cc:1905] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:53.349457 30690 ts_tablet_manager.cc:1905] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:53.349998 30689 ts_tablet_manager.cc:1918] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:23:53.350114 30689 log.cc:1196] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/1dd3796e733f4e429a5365e15ec55e87
I20260619 08:23:53.350440 30689 ts_tablet_manager.cc:1939] T 1dd3796e733f4e429a5365e15ec55e87 P 745faba18e5d425398090711941b2ca9: Deleting consensus metadata
I20260619 08:23:53.350979 30180 catalog_manager.cc:4841] TS 745faba18e5d425398090711941b2ca9 (127.25.127.131:42975): tablet 1dd3796e733f4e429a5365e15ec55e87 (table test-table [id=1caca0c0a81e4e9d9f57a9c55155d375]) successfully deleted
I20260619 08:23:53.351460 30690 ts_tablet_manager.cc:1918] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:23:53.351539 30690 log.cc:1196] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/1dd3796e733f4e429a5365e15ec55e87
I20260619 08:23:53.351804 30690 ts_tablet_manager.cc:1939] T 1dd3796e733f4e429a5365e15ec55e87 P 56aed3e7c81445308d5696956f6f61d6: Deleting consensus metadata
I20260619 08:23:53.352326 30179 catalog_manager.cc:4841] TS 56aed3e7c81445308d5696956f6f61d6 (127.25.127.130:42139): tablet 1dd3796e733f4e429a5365e15ec55e87 (table test-table [id=1caca0c0a81e4e9d9f57a9c55155d375]) successfully deleted
W20260619 08:23:53.353767 30177 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.127.129:46707 (error 108)
I20260619 08:23:53.354100 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 30388
I20260619 08:23:53.354305 30660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.354166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:46707 (local address 127.0.0.1:57442)
0619 08:23:53.354229 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:53.354265 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.129:46707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 08:23:53.356299 30177 catalog_manager.cc:4584] TS dce51ea010734d78801a2032e3ca0339 (127.25.127.129:46707): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1dd3796e733f4e429a5365e15ec55e87: Network error: Client connection negotiation failed: client connection to 127.25.127.129:46707: connect: Connection refused (error 111)
I20260619 08:23:53.359373 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 30521
I20260619 08:23:53.363754 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 30162
2026-06-19T08:23:53Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2043 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T08:23:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:23:53Z Disabled control of system clock
I20260619 08:23:53.391556 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:40051
--webserver_interface=127.25.127.190
--webserver_port=0
--builtin_ntp_servers=127.25.127.148:41389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:40051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 08:23:53.478494 30695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:53.478721 30695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:53.478749 30695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:53.479861 30695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:23:53.479897 30695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:53.479908 30695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:53.479921 30695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:23:53.479933 30695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:23:53.481389 30695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:41389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:40051
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:40051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:53.481619 30695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:53.481858 30695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:53.486897 30700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:53.486903 30701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:53.486903 30703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:53.487130 30695 server_base.cc:1034] running on GCE node
I20260619 08:23:53.487442 30695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:53.487737 30695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:53.488924 30695 hybrid_clock.cc:648] HybridClock initialized: now 1781857433488911 us; error 47 us; skew 500 ppm
I20260619 08:23:53.490216 30695 webserver.cc:460] Webserver started at http://127.25.127.190:40069/ using document root <none> and password file <none>
I20260619 08:23:53.490469 30695 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:53.490540 30695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:53.490688 30695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:53.491832 30695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "21a4aeb1aa894dbeaa018bba7fd30de3"
format_stamp: "Formatted at 2026-06-19 08:23:53 on dist-test-slave-5rhg"
server_key: "4c01a63541825b907a13e635b78beed0"
server_key_iv: "46ff68d5ed2f18aad5f1932b400f6f52"
server_key_version: "encryptionkey@0"
I20260619 08:23:53.492262 30695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "21a4aeb1aa894dbeaa018bba7fd30de3"
format_stamp: "Formatted at 2026-06-19 08:23:53 on dist-test-slave-5rhg"
server_key: "4c01a63541825b907a13e635b78beed0"
server_key_iv: "46ff68d5ed2f18aad5f1932b400f6f52"
server_key_version: "encryptionkey@0"
I20260619 08:23:53.493727 30695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:53.494545 30709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:53.494789 30695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:53.494890 30695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "21a4aeb1aa894dbeaa018bba7fd30de3"
format_stamp: "Formatted at 2026-06-19 08:23:53 on dist-test-slave-5rhg"
server_key: "4c01a63541825b907a13e635b78beed0"
server_key_iv: "46ff68d5ed2f18aad5f1932b400f6f52"
server_key_version: "encryptionkey@0"
I20260619 08:23:53.494974 30695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:53.498805 30695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:53.499143 30695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:53.499300 30695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:53.503015 30695 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:40051
I20260619 08:23:53.503090 30761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:40051 every 8 connection(s)
I20260619 08:23:53.503425 30695 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:23:53.504225 30762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:53.505826 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 30695
I20260619 08:23:53.505939 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:23:53.506098 26110 external_mini_cluster.cc:1428] Setting key 662b8c1f6ba871ba5039cc1f9da1c4fa
I20260619 08:23:53.508478 30762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3: Bootstrap starting.
I20260619 08:23:53.508846 30765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.506681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42058 (local address 127.25.127.190:40051)
0619 08:23:53.506887 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.506897 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.506931 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:23:53.507505 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.507515 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.507573 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.507730 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:53.508372 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:53.508376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:53.508377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:23:53.508381 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:53.508426 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:23:53.508440 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:53.508517 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.508605 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.508662 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260619 08:23:53.509246 30762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:53.509543 30762 log.cc:824] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:53.510255 30762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3: No bootstrap required, opened a new log
I20260619 08:23:53.511663 30762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a4aeb1aa894dbeaa018bba7fd30de3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40051 } }
I20260619 08:23:53.511833 30762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:53.511881 30762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a4aeb1aa894dbeaa018bba7fd30de3, State: Initialized, Role: FOLLOWER
I20260619 08:23:53.512002 30762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a4aeb1aa894dbeaa018bba7fd30de3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40051 } }
I20260619 08:23:53.512115 30762 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:23:53.512177 30762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:23:53.512231 30762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:53.512795 30762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a4aeb1aa894dbeaa018bba7fd30de3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40051 } }
I20260619 08:23:53.512925 30762 leader_election.cc:302] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21a4aeb1aa894dbeaa018bba7fd30de3; no voters:
I20260619 08:23:53.513126 30762 leader_election.cc:288] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:23:53.513410 30762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:53.513476 30767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:53.513640 30767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3 [term 1 LEADER]: Becoming Leader. State: Replica: 21a4aeb1aa894dbeaa018bba7fd30de3, State: Running, Role: LEADER
I20260619 08:23:53.513749 30767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a4aeb1aa894dbeaa018bba7fd30de3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40051 } }
I20260619 08:23:53.514168 30768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21a4aeb1aa894dbeaa018bba7fd30de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a4aeb1aa894dbeaa018bba7fd30de3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40051 } } }
I20260619 08:23:53.514261 30768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3 [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:53.514416 30769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21a4aeb1aa894dbeaa018bba7fd30de3. Latest consensus state: current_term: 1 leader_uuid: "21a4aeb1aa894dbeaa018bba7fd30de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a4aeb1aa894dbeaa018bba7fd30de3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40051 } } }
I20260619 08:23:53.514484 30776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:23:53.514496 30769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3 [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:53.515309 30776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:23:53.516913 30776 catalog_manager.cc:1347] Generated new cluster ID: 2c57813fb55e4419ab44f9a7d965b36a
I20260619 08:23:53.516966 30776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:23:53.551151 30776 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:23:53.551375 30776 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:23:53.558877 30776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3: Generated new TSK 0
I20260619 08:23:53.559129 30776 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:23:53.560892 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:40051
--builtin_ntp_servers=127.25.127.148:41389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 08:23:53.646413 30786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:53.646684 30786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:53.646726 30786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:53.647974 30786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:53.648038 30786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:53.648082 30786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:23:53.649466 30786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:41389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:40051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:53.649755 30786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:53.650066 30786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:53.655294 30794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:53.655381 30791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:53.655306 30792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:53.655611 30786 server_base.cc:1034] running on GCE node
I20260619 08:23:53.655871 30786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:53.656188 30786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:53.657353 30786 hybrid_clock.cc:648] HybridClock initialized: now 1781857433657345 us; error 36 us; skew 500 ppm
I20260619 08:23:53.658666 30786 webserver.cc:460] Webserver started at http://127.25.127.129:33085/ using document root <none> and password file <none>
I20260619 08:23:53.658915 30786 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:53.658985 30786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:53.659122 30786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:53.660339 30786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "cafaf7b06c3d4b16b8c0c78d2fb663a9"
format_stamp: "Formatted at 2026-06-19 08:23:53 on dist-test-slave-5rhg"
server_key: "f2acf16a4d5091017f9a22efc2758bd0"
server_key_iv: "6c4e5c0df689a846214a3bd83524ce8d"
server_key_version: "encryptionkey@0"
I20260619 08:23:53.660709 30786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "cafaf7b06c3d4b16b8c0c78d2fb663a9"
format_stamp: "Formatted at 2026-06-19 08:23:53 on dist-test-slave-5rhg"
server_key: "f2acf16a4d5091017f9a22efc2758bd0"
server_key_iv: "6c4e5c0df689a846214a3bd83524ce8d"
server_key_version: "encryptionkey@0"
I20260619 08:23:53.662067 30786 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:23:53.662834 30800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:53.663019 30786 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:53.663120 30786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "cafaf7b06c3d4b16b8c0c78d2fb663a9"
format_stamp: "Formatted at 2026-06-19 08:23:53 on dist-test-slave-5rhg"
server_key: "f2acf16a4d5091017f9a22efc2758bd0"
server_key_iv: "6c4e5c0df689a846214a3bd83524ce8d"
server_key_version: "encryptionkey@0"
I20260619 08:23:53.663213 30786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:53.666704 30786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:53.666970 30786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:53.667115 30786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:53.667338 30786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:53.667649 30786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:53.667702 30786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:53.667744 30786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:53.667771 30786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:53.673575 30786 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:46285
I20260619 08:23:53.673661 30913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:46285 every 8 connection(s)
I20260619 08:23:53.674073 30786 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:23:53.675767 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 30786
I20260619 08:23:53.675875 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:23:53.676086 26110 external_mini_cluster.cc:1428] Setting key d886db40677abb2b55b008c5e85fa1fa
I20260619 08:23:53.677251 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:40051
--builtin_ntp_servers=127.25.127.148:41389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 08:23:53.677703 30765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.674858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:34535 (local address 127.25.127.190:40051)
0619 08:23:53.674903 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.674906 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.675118 (+ 212us) server_negotiation.cc:407] Connection header received
0619 08:23:53.675644 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.675645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.675651 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.675718 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:53.677293 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:53.677294 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:53.677295 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:23:53.677335 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:53.677355 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:23:53.677360 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:53.677455 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.677580 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.677598 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:23:53.677863 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.674851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:40051 (local address 127.25.127.129:34535)
0619 08:23:53.675018 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:53.675050 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:53.675518 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:53.675800 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:53.675805 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:53.675815 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:53.677070 (+ 1255us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:23:53.677070 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:53.677112 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:23:53.677149 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:53.677489 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:53.677497 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:53.677500 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:23:53.677581 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 08:23:53.677638 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260619 08:23:53.678387 30914 heartbeater.cc:347] Connected to a master server at 127.25.127.190:40051
I20260619 08:23:53.678498 30914 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:53.678674 30914 heartbeater.cc:511] Master 127.25.127.190:40051 requested a full tablet report, sending...
I20260619 08:23:53.679203 30726 ts_manager.cc:194] Registered new tserver with Master: cafaf7b06c3d4b16b8c0c78d2fb663a9 (127.25.127.129:46285)
W20260619 08:23:53.761605 30917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:53.761823 30917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:53.761849 30917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:53.763058 30917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:53.763100 30917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:53.763129 30917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:23:53.764566 30917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:41389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:40051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:53.764776 30917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:53.765030 30917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:53.770465 30923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:53.770506 30925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:53.770598 30917 server_base.cc:1034] running on GCE node
W20260619 08:23:53.770462 30922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:53.770887 30917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:53.771157 30917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:53.772337 30917 hybrid_clock.cc:648] HybridClock initialized: now 1781857433772324 us; error 26 us; skew 500 ppm
I20260619 08:23:53.773550 30917 webserver.cc:460] Webserver started at http://127.25.127.130:34031/ using document root <none> and password file <none>
I20260619 08:23:53.773820 30917 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:53.773892 30917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:53.774044 30917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:53.775141 30917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "e9e0983c788240c99d02aff0baf05169"
format_stamp: "Formatted at 2026-06-19 08:23:53 on dist-test-slave-5rhg"
server_key: "5ce15d4bdbdc44ef543fff52233e818a"
server_key_iv: "58d416d89572c51856dc4a774507f760"
server_key_version: "encryptionkey@0"
I20260619 08:23:53.775511 30917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "e9e0983c788240c99d02aff0baf05169"
format_stamp: "Formatted at 2026-06-19 08:23:53 on dist-test-slave-5rhg"
server_key: "5ce15d4bdbdc44ef543fff52233e818a"
server_key_iv: "58d416d89572c51856dc4a774507f760"
server_key_version: "encryptionkey@0"
I20260619 08:23:53.776932 30917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:23:53.777716 30931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:53.777915 30917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:53.778021 30917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "e9e0983c788240c99d02aff0baf05169"
format_stamp: "Formatted at 2026-06-19 08:23:53 on dist-test-slave-5rhg"
server_key: "5ce15d4bdbdc44ef543fff52233e818a"
server_key_iv: "58d416d89572c51856dc4a774507f760"
server_key_version: "encryptionkey@0"
I20260619 08:23:53.778100 30917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:53.781564 30917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:53.781847 30917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:53.781989 30917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:53.782241 30917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:53.782539 30917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:53.782598 30917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:53.782644 30917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:53.782670 30917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:53.789000 30917 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:36539
I20260619 08:23:53.789040 31044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:36539 every 8 connection(s)
I20260619 08:23:53.789372 30917 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:23:53.792364 30765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.790178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:52449 (local address 127.25.127.190:40051)
0619 08:23:53.790309 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.790311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.790463 (+ 152us) server_negotiation.cc:407] Connection header received
0619 08:23:53.790938 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.790938 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.790943 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.791013 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:53.792100 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:53.792101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:53.792102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:23:53.792106 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:53.792124 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:23:53.792128 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:53.792240 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.792296 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.792308 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260619 08:23:53.792513 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 30917
I20260619 08:23:53.792627 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:23:53.792629 31047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.790144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:40051 (local address 127.25.127.130:52449)
0619 08:23:53.790332 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:53.790372 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:53.790853 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:53.791067 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:53.791070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:53.791078 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:53.791958 (+ 880us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:23:53.791958 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:53.791987 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:23:53.792015 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:53.792242 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:53.792245 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:53.792248 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:23:53.792320 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 08:23:53.792391 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260619 08:23:53.792822 26110 external_mini_cluster.cc:1428] Setting key 76cb7761f1f66ec57e15d5780914aba0
I20260619 08:23:53.793376 31045 heartbeater.cc:347] Connected to a master server at 127.25.127.190:40051
I20260619 08:23:53.793485 31045 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:53.793637 31045 heartbeater.cc:511] Master 127.25.127.190:40051 requested a full tablet report, sending...
I20260619 08:23:53.794068 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:40051
--builtin_ntp_servers=127.25.127.148:41389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 08:23:53.794071 30725 ts_manager.cc:194] Registered new tserver with Master: e9e0983c788240c99d02aff0baf05169 (127.25.127.130:36539)
W20260619 08:23:53.875785 31048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:53.876008 31048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:53.876031 31048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:53.877265 31048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:53.877312 31048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:53.877339 31048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:23:53.878690 31048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:41389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:40051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:53.878965 31048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:53.879298 31048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:53.884506 31053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:53.884533 31054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:53.884542 31056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:53.884904 31048 server_base.cc:1034] running on GCE node
I20260619 08:23:53.885128 31048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:53.885442 31048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:53.886590 31048 hybrid_clock.cc:648] HybridClock initialized: now 1781857433886573 us; error 34 us; skew 500 ppm
I20260619 08:23:53.887833 31048 webserver.cc:460] Webserver started at http://127.25.127.131:35671/ using document root <none> and password file <none>
I20260619 08:23:53.888096 31048 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:53.888197 31048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:53.888371 31048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:53.889454 31048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "1235cb9da5244d1eb63025cda785c082"
format_stamp: "Formatted at 2026-06-19 08:23:53 on dist-test-slave-5rhg"
server_key: "9dc0d0cd780dee453c3f92b783844a45"
server_key_iv: "550ec6f4e8cf1d14f6ae75fe92486b97"
server_key_version: "encryptionkey@0"
I20260619 08:23:53.889824 31048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "1235cb9da5244d1eb63025cda785c082"
format_stamp: "Formatted at 2026-06-19 08:23:53 on dist-test-slave-5rhg"
server_key: "9dc0d0cd780dee453c3f92b783844a45"
server_key_iv: "550ec6f4e8cf1d14f6ae75fe92486b97"
server_key_version: "encryptionkey@0"
I20260619 08:23:53.891309 31048 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:23:53.892078 31062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:53.892300 31048 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:53.892400 31048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "1235cb9da5244d1eb63025cda785c082"
format_stamp: "Formatted at 2026-06-19 08:23:53 on dist-test-slave-5rhg"
server_key: "9dc0d0cd780dee453c3f92b783844a45"
server_key_iv: "550ec6f4e8cf1d14f6ae75fe92486b97"
server_key_version: "encryptionkey@0"
I20260619 08:23:53.892488 31048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:53.895833 31048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:53.896101 31048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:53.896270 31048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:53.896487 31048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:53.896796 31048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:53.896852 31048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:53.896891 31048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:53.896919 31048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:53.902853 31048 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:35029
I20260619 08:23:53.902887 31175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:35029 every 8 connection(s)
I20260619 08:23:53.903239 31048 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:23:53.906176 30765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.904061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:43411 (local address 127.25.127.190:40051)
0619 08:23:53.904139 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.904141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.904344 (+ 203us) server_negotiation.cc:407] Connection header received
0619 08:23:53.904804 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.904805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.904810 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.904876 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:53.905964 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:53.905965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:53.905966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:23:53.905969 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:53.905984 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:23:53.905988 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:53.906041 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.906110 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.906122 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 08:23:53.906406 31178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.903985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:40051 (local address 127.25.127.131:43411)
0619 08:23:53.904217 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:53.904258 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:53.904720 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:53.904938 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:53.904941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:53.904949 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:53.905824 (+ 875us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:23:53.905824 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:53.905852 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:23:53.905880 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:53.906042 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:53.906044 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:53.906047 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:23:53.906099 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:23:53.906174 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260619 08:23:53.906873 31176 heartbeater.cc:347] Connected to a master server at 127.25.127.190:40051
I20260619 08:23:53.906965 31176 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:53.907166 31176 heartbeater.cc:511] Master 127.25.127.190:40051 requested a full tablet report, sending...
I20260619 08:23:53.907538 30725 ts_manager.cc:194] Registered new tserver with Master: 1235cb9da5244d1eb63025cda785c082 (127.25.127.131:35029)
I20260619 08:23:53.909003 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 31048
I20260619 08:23:53.909133 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:23:53.909283 26110 external_mini_cluster.cc:1428] Setting key b7eafae75227c46f1615b89da9ae606f
I20260619 08:23:53.910043 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:23:53.912420 30765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.911388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42066 (local address 127.25.127.190:40051)
0619 08:23:53.911459 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.911461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.911517 (+ 56us) server_negotiation.cc:407] Connection header received
0619 08:23:53.911567 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.911568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.911572 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.911610 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:53.912138 (+ 528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:53.912139 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:53.912139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:23:53.912143 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:53.912199 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:23:53.912203 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:53.912265 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.912304 (+ 39us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.912315 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 08:23:53.914100 30725 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:23:53.915696 30725 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:23:53.920295 31187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.918204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39858 (local address 127.25.127.130:36539)
0619 08:23:53.918730 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.918734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.918743 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:23:53.918766 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.918768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.918801 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.918884 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:53.919802 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:53.919814 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:53.919814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:23:53.919820 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:53.919859 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:23:53.919898 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:53.919935 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.920129 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.920151 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":114,"threads_started":1}
I20260619 08:23:53.920315 31184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.918188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:46285 (local address 127.0.0.1:52696)
0619 08:23:53.918405 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:53.918429 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:53.918492 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:53.919481 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:53.919482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:53.919485 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:53.919858 (+ 373us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:23:53.919859 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:53.919868 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:23:53.919883 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:53.920171 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:53.920172 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:53.920173 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:23:53.920226 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 08:23:53.920240 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":110,"threads_started":1}
I20260619 08:23:53.920552 31185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.918112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:36539 (local address 127.0.0.1:39858)
0619 08:23:53.918378 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:53.918429 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:53.918492 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:53.918983 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:53.918992 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:53.918999 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:53.919642 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:23:53.919642 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:53.919661 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:23:53.919687 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:53.920001 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:53.920009 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:53.920012 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:23:53.920438 (+ 426us) client_negotiation.cc:240] Negotiation successful
0619 08:23:53.920454 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":85,"threads_started":1}
I20260619 08:23:53.920646 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.918442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52696 (local address 127.25.127.129:46285)
0619 08:23:53.919323 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.919329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.919339 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:23:53.919365 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.919367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.919376 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.919479 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:53.919983 (+ 504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:53.919991 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:53.919991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:23:53.919996 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:53.920033 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:23:53.920058 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:53.920120 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.920541 (+ 421us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.920559 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":834,"thread_start_us":81,"threads_started":1}
I20260619 08:23:53.920996 31186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.918190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:35029 (local address 127.0.0.1:47466)
0619 08:23:53.918584 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:53.918593 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:53.918633 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:53.920025 (+ 1392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:53.920026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:53.920036 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:53.920531 (+ 495us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:23:53.920531 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:53.920537 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:23:53.920549 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:53.920864 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:53.920865 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:53.920865 (+ 0us) client_negotiation.cc:769] Sending connection context
0619 08:23:53.920900 (+ 35us) client_negotiation.cc:240] Negotiation successful
0619 08:23:53.920932 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":113,"threads_started":1}
I20260619 08:23:53.921327 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.918439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47466 (local address 127.25.127.131:35029)
0619 08:23:53.919864 (+ 1425us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.919869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.919880 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:23:53.919906 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.919909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.919918 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.920009 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:53.920655 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:53.920664 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:53.920664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:23:53.920670 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:53.920714 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:23:53.920767 (+ 53us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:53.921190 (+ 423us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.921218 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.921238 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1382,"thread_start_us":89,"threads_started":1}
I20260619 08:23:53.921468 30979 tablet_service.cc:1467] Processing CreateTablet for tablet c726da3becd04ffa98b2afdfca6b34d8 (DEFAULT_TABLE table=test-table [id=6495a9032eba4818aed29a48fcae69b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:53.921481 30848 tablet_service.cc:1467] Processing CreateTablet for tablet c726da3becd04ffa98b2afdfca6b34d8 (DEFAULT_TABLE table=test-table [id=6495a9032eba4818aed29a48fcae69b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:53.921872 30979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c726da3becd04ffa98b2afdfca6b34d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:53.921885 30848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c726da3becd04ffa98b2afdfca6b34d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:53.924500 31190 tablet_bootstrap.cc:492] T c726da3becd04ffa98b2afdfca6b34d8 P e9e0983c788240c99d02aff0baf05169: Bootstrap starting.
I20260619 08:23:53.924628 31110 tablet_service.cc:1467] Processing CreateTablet for tablet c726da3becd04ffa98b2afdfca6b34d8 (DEFAULT_TABLE table=test-table [id=6495a9032eba4818aed29a48fcae69b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:53.924873 31191 tablet_bootstrap.cc:492] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9: Bootstrap starting.
I20260619 08:23:53.924986 31110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c726da3becd04ffa98b2afdfca6b34d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:53.925277 31190 tablet_bootstrap.cc:654] T c726da3becd04ffa98b2afdfca6b34d8 P e9e0983c788240c99d02aff0baf05169: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:53.925546 31191 tablet_bootstrap.cc:654] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:53.925621 31190 log.cc:824] T c726da3becd04ffa98b2afdfca6b34d8 P e9e0983c788240c99d02aff0baf05169: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:53.925879 31191 log.cc:824] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:53.926436 31190 tablet_bootstrap.cc:492] T c726da3becd04ffa98b2afdfca6b34d8 P e9e0983c788240c99d02aff0baf05169: No bootstrap required, opened a new log
I20260619 08:23:53.926524 31190 ts_tablet_manager.cc:1397] T c726da3becd04ffa98b2afdfca6b34d8 P e9e0983c788240c99d02aff0baf05169: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:23:53.927930 31194 tablet_bootstrap.cc:492] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082: Bootstrap starting.
I20260619 08:23:53.928815 31194 tablet_bootstrap.cc:654] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:53.928824 31191 tablet_bootstrap.cc:492] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9: No bootstrap required, opened a new log
I20260619 08:23:53.928915 31191 ts_tablet_manager.cc:1397] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 08:23:53.928932 31190 raft_consensus.cc:348] T c726da3becd04ffa98b2afdfca6b34d8 P e9e0983c788240c99d02aff0baf05169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e0983c788240c99d02aff0baf05169" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36539 } } peers { permanent_uuid: "cafaf7b06c3d4b16b8c0c78d2fb663a9" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46285 } } peers { permanent_uuid: "1235cb9da5244d1eb63025cda785c082" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35029 } }
I20260619 08:23:53.929137 31190 raft_consensus.cc:374] T c726da3becd04ffa98b2afdfca6b34d8 P e9e0983c788240c99d02aff0baf05169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:53.929179 31190 raft_consensus.cc:729] T c726da3becd04ffa98b2afdfca6b34d8 P e9e0983c788240c99d02aff0baf05169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9e0983c788240c99d02aff0baf05169, State: Initialized, Role: FOLLOWER
I20260619 08:23:53.929183 31194 log.cc:824] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:53.929353 31190 consensus_queue.cc:260] T c726da3becd04ffa98b2afdfca6b34d8 P e9e0983c788240c99d02aff0baf05169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e0983c788240c99d02aff0baf05169" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36539 } } peers { permanent_uuid: "cafaf7b06c3d4b16b8c0c78d2fb663a9" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46285 } } peers { permanent_uuid: "1235cb9da5244d1eb63025cda785c082" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35029 } }
I20260619 08:23:53.929701 31190 ts_tablet_manager.cc:1428] T c726da3becd04ffa98b2afdfca6b34d8 P e9e0983c788240c99d02aff0baf05169: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:23:53.929845 31045 heartbeater.cc:503] Master 127.25.127.190:40051 was elected leader, sending a full tablet report...
I20260619 08:23:53.930025 31194 tablet_bootstrap.cc:492] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082: No bootstrap required, opened a new log
I20260619 08:23:53.930263 31194 ts_tablet_manager.cc:1397] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:23:53.931768 31191 raft_consensus.cc:348] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e0983c788240c99d02aff0baf05169" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36539 } } peers { permanent_uuid: "cafaf7b06c3d4b16b8c0c78d2fb663a9" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46285 } } peers { permanent_uuid: "1235cb9da5244d1eb63025cda785c082" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35029 } }
I20260619 08:23:53.931926 31191 raft_consensus.cc:374] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:53.931967 31191 raft_consensus.cc:729] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafaf7b06c3d4b16b8c0c78d2fb663a9, State: Initialized, Role: FOLLOWER
I20260619 08:23:53.932092 31191 consensus_queue.cc:260] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e0983c788240c99d02aff0baf05169" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36539 } } peers { permanent_uuid: "cafaf7b06c3d4b16b8c0c78d2fb663a9" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46285 } } peers { permanent_uuid: "1235cb9da5244d1eb63025cda785c082" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35029 } }
I20260619 08:23:53.932379 31191 ts_tablet_manager.cc:1428] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 08:23:53.932471 30914 heartbeater.cc:503] Master 127.25.127.190:40051 was elected leader, sending a full tablet report...
I20260619 08:23:53.932652 31194 raft_consensus.cc:348] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e0983c788240c99d02aff0baf05169" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36539 } } peers { permanent_uuid: "cafaf7b06c3d4b16b8c0c78d2fb663a9" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46285 } } peers { permanent_uuid: "1235cb9da5244d1eb63025cda785c082" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35029 } }
I20260619 08:23:53.932869 31194 raft_consensus.cc:374] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:53.932947 31194 raft_consensus.cc:729] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1235cb9da5244d1eb63025cda785c082, State: Initialized, Role: FOLLOWER
I20260619 08:23:53.933079 31194 consensus_queue.cc:260] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e0983c788240c99d02aff0baf05169" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36539 } } peers { permanent_uuid: "cafaf7b06c3d4b16b8c0c78d2fb663a9" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46285 } } peers { permanent_uuid: "1235cb9da5244d1eb63025cda785c082" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35029 } }
I20260619 08:23:53.933357 31194 ts_tablet_manager.cc:1428] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:23:53.933457 31176 heartbeater.cc:503] Master 127.25.127.190:40051 was elected leader, sending a full tablet report...
I20260619 08:23:53.986663 31197 raft_consensus.cc:482] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:23:53.986831 31197 raft_consensus.cc:504] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e0983c788240c99d02aff0baf05169" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36539 } } peers { permanent_uuid: "cafaf7b06c3d4b16b8c0c78d2fb663a9" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46285 } } peers { permanent_uuid: "1235cb9da5244d1eb63025cda785c082" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35029 } }
I20260619 08:23:53.987289 31197 leader_election.cc:288] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9e0983c788240c99d02aff0baf05169 (127.25.127.130:36539), 1235cb9da5244d1eb63025cda785c082 (127.25.127.131:35029)
I20260619 08:23:53.988883 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.987533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:36539 (local address 127.25.127.129:34225)
0619 08:23:53.987614 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:53.987630 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:53.987692 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:53.987851 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:53.987852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:53.987855 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:53.988464 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:23:53.988465 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:53.988472 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:23:53.988487 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:53.988701 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:53.988701 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:53.988702 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:23:53.988786 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 08:23:53.988801 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 08:23:53.988917 31187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.987657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:34225 (local address 127.25.127.130:36539)
0619 08:23:53.987714 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.987717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.987729 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:23:53.987762 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.987763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.987771 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.987831 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:53.988570 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:53.988587 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:53.988587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:23:53.988592 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:53.988625 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:23:53.988631 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:53.988697 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.988799 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.988812 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 08:23:53.989013 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.987533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:35029 (local address 127.25.127.129:60779)
0619 08:23:53.987746 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:53.987756 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:53.987809 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:53.987945 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:53.987946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:53.987948 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:53.988657 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:23:53.988657 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:53.988663 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:23:53.988677 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:53.988866 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:53.988867 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:53.988868 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:23:53.988918 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 08:23:53.988932 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":17,"thread_start_us":114,"threads_started":1}
I20260619 08:23:53.989009 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.987723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:60779 (local address 127.25.127.131:35029)
0619 08:23:53.987772 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.987774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.987803 (+ 29us) server_negotiation.cc:407] Connection header received
0619 08:23:53.987865 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.987866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.987871 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.987931 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:53.988767 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:53.988769 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:53.988770 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:23:53.988775 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:53.988794 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:23:53.988799 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:53.988853 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.988916 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.988930 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:23:53.989326 30999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c726da3becd04ffa98b2afdfca6b34d8" candidate_uuid: "cafaf7b06c3d4b16b8c0c78d2fb663a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9e0983c788240c99d02aff0baf05169" is_pre_election: true
I20260619 08:23:53.989481 30999 raft_consensus.cc:2454] T c726da3becd04ffa98b2afdfca6b34d8 P e9e0983c788240c99d02aff0baf05169 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafaf7b06c3d4b16b8c0c78d2fb663a9 in term 0.
I20260619 08:23:53.989724 30803 leader_election.cc:302] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cafaf7b06c3d4b16b8c0c78d2fb663a9, e9e0983c788240c99d02aff0baf05169; no voters:
I20260619 08:23:53.989748 31130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c726da3becd04ffa98b2afdfca6b34d8" candidate_uuid: "cafaf7b06c3d4b16b8c0c78d2fb663a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1235cb9da5244d1eb63025cda785c082" is_pre_election: true
I20260619 08:23:53.989876 31197 raft_consensus.cc:2781] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:23:53.989883 31130 raft_consensus.cc:2454] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cafaf7b06c3d4b16b8c0c78d2fb663a9 in term 0.
I20260619 08:23:53.989936 31197 raft_consensus.cc:482] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:23:53.989972 31197 raft_consensus.cc:3037] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:53.990748 31197 raft_consensus.cc:504] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e0983c788240c99d02aff0baf05169" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36539 } } peers { permanent_uuid: "cafaf7b06c3d4b16b8c0c78d2fb663a9" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46285 } } peers { permanent_uuid: "1235cb9da5244d1eb63025cda785c082" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35029 } }
I20260619 08:23:53.990950 31197 leader_election.cc:288] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [CANDIDATE]: Term 1 election: Requested vote from peers e9e0983c788240c99d02aff0baf05169 (127.25.127.130:36539), 1235cb9da5244d1eb63025cda785c082 (127.25.127.131:35029)
I20260619 08:23:53.991104 30999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c726da3becd04ffa98b2afdfca6b34d8" candidate_uuid: "cafaf7b06c3d4b16b8c0c78d2fb663a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9e0983c788240c99d02aff0baf05169"
I20260619 08:23:53.991133 31130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c726da3becd04ffa98b2afdfca6b34d8" candidate_uuid: "cafaf7b06c3d4b16b8c0c78d2fb663a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1235cb9da5244d1eb63025cda785c082"
I20260619 08:23:53.991191 30999 raft_consensus.cc:3037] T c726da3becd04ffa98b2afdfca6b34d8 P e9e0983c788240c99d02aff0baf05169 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:53.991202 31130 raft_consensus.cc:3037] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:53.992138 30999 raft_consensus.cc:2454] T c726da3becd04ffa98b2afdfca6b34d8 P e9e0983c788240c99d02aff0baf05169 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafaf7b06c3d4b16b8c0c78d2fb663a9 in term 1.
I20260619 08:23:53.992138 31130 raft_consensus.cc:2454] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cafaf7b06c3d4b16b8c0c78d2fb663a9 in term 1.
I20260619 08:23:53.992305 30803 leader_election.cc:302] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cafaf7b06c3d4b16b8c0c78d2fb663a9, e9e0983c788240c99d02aff0baf05169; no voters:
I20260619 08:23:53.992406 31197 raft_consensus.cc:2781] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:53.992611 31197 raft_consensus.cc:686] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [term 1 LEADER]: Becoming Leader. State: Replica: cafaf7b06c3d4b16b8c0c78d2fb663a9, State: Running, Role: LEADER
I20260619 08:23:53.992745 31197 consensus_queue.cc:237] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e0983c788240c99d02aff0baf05169" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36539 } } peers { permanent_uuid: "cafaf7b06c3d4b16b8c0c78d2fb663a9" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46285 } } peers { permanent_uuid: "1235cb9da5244d1eb63025cda785c082" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35029 } }
I20260619 08:23:53.993657 30725 catalog_manager.cc:5495] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to cafaf7b06c3d4b16b8c0c78d2fb663a9 (127.25.127.129). New cstate: current_term: 1 leader_uuid: "cafaf7b06c3d4b16b8c0c78d2fb663a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9e0983c788240c99d02aff0baf05169" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cafaf7b06c3d4b16b8c0c78d2fb663a9" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1235cb9da5244d1eb63025cda785c082" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35029 } health_report { overall_health: UNKNOWN } } }
I20260619 08:23:53.999141 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:53.998136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52700 (local address 127.25.127.129:46285)
0619 08:23:53.998194 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:53.998198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:53.998209 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:23:53.998241 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:53.998241 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:53.998253 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:53.998319 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:53.998912 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:53.998913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:53.998914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:23:53.998918 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:53.998951 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:23:53.998956 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:53.999017 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:53.999081 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 08:23:53.999092 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:23:54.001060 31130 raft_consensus.cc:1261] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082 [term 1 FOLLOWER]: Refusing update from remote peer cafaf7b06c3d4b16b8c0c78d2fb663a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:54.001086 30999 raft_consensus.cc:1261] T c726da3becd04ffa98b2afdfca6b34d8 P e9e0983c788240c99d02aff0baf05169 [term 1 FOLLOWER]: Refusing update from remote peer cafaf7b06c3d4b16b8c0c78d2fb663a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:23:54.001466 31197 consensus_queue.cc:1035] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9e0983c788240c99d02aff0baf05169" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:54.001592 31200 consensus_queue.cc:1035] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1235cb9da5244d1eb63025cda785c082" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:54.003964 31203 mvcc.cc:205] Tried to move back new op lower bound from 7298488049665589248 to 7298488049635041280. Current Snapshot: MvccSnapshot[applied={T|T < 7298488049665589248}]
I20260619 08:23:54.006927 30725 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:23:54.007021 30725 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:23:54.007773 30725 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21a4aeb1aa894dbeaa018bba7fd30de3: Sending DeleteTablet for 3 replicas of tablet c726da3becd04ffa98b2afdfca6b34d8
I20260619 08:23:54.008143 30848 tablet_service.cc:1514] Processing DeleteTablet for tablet c726da3becd04ffa98b2afdfca6b34d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:54 UTC) from {username='slave'} at 127.0.0.1:52696
I20260619 08:23:54.008188 31110 tablet_service.cc:1514] Processing DeleteTablet for tablet c726da3becd04ffa98b2afdfca6b34d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:54 UTC) from {username='slave'} at 127.0.0.1:47466
I20260619 08:23:54.008186 30979 tablet_service.cc:1514] Processing DeleteTablet for tablet c726da3becd04ffa98b2afdfca6b34d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:23:54 UTC) from {username='slave'} at 127.0.0.1:39858
I20260619 08:23:54.008392 31213 tablet_replica.cc:330] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9: stopping tablet replica
I20260619 08:23:54.008486 31213 raft_consensus.cc:2229] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:23:54.008599 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 30786
I20260619 08:23:54.008610 31213 raft_consensus.cc:2258] T c726da3becd04ffa98b2afdfca6b34d8 P cafaf7b06c3d4b16b8c0c78d2fb663a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:54.008750 31214 tablet_replica.cc:330] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082: stopping tablet replica
I20260619 08:23:54.008831 31214 raft_consensus.cc:2229] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:23:54.008900 31214 raft_consensus.cc:2258] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:54.010151 31214 ts_tablet_manager.cc:1905] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:54.012171 31214 ts_tablet_manager.cc:1918] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:23:54.012251 31214 log.cc:1196] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/c726da3becd04ffa98b2afdfca6b34d8
I20260619 08:23:54.012508 31214 ts_tablet_manager.cc:1939] T c726da3becd04ffa98b2afdfca6b34d8 P 1235cb9da5244d1eb63025cda785c082: Deleting consensus metadata
I20260619 08:23:54.012910 30713 catalog_manager.cc:4841] TS 1235cb9da5244d1eb63025cda785c082 (127.25.127.131:35029): tablet c726da3becd04ffa98b2afdfca6b34d8 (table test-table [id=6495a9032eba4818aed29a48fcae69b4]) successfully deleted
W20260619 08:23:54.015076 30711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.127.129:46285 (error 108)
I20260619 08:23:54.015321 31215 tablet_replica.cc:330] T c726da3becd04ffa98b2afdfca6b34d8 P e9e0983c788240c99d02aff0baf05169: stopping tablet replica
I20260619 08:23:54.015443 31215 raft_consensus.cc:2229] T c726da3becd04ffa98b2afdfca6b34d8 P e9e0983c788240c99d02aff0baf05169 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:23:54.015569 31186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:54.015427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:46285 (local address 127.0.0.1:52712)
0619 08:23:54.015467 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:54.015534 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.129:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260619 08:23:54.015591 31215 raft_consensus.cc:2258] T c726da3becd04ffa98b2afdfca6b34d8 P e9e0983c788240c99d02aff0baf05169 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 08:23:54.015703 30711 catalog_manager.cc:4584] TS cafaf7b06c3d4b16b8c0c78d2fb663a9 (127.25.127.129:46285): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c726da3becd04ffa98b2afdfca6b34d8: Network error: Client connection negotiation failed: client connection to 127.25.127.129:46285: connect: Connection refused (error 111)
I20260619 08:23:54.015770 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 30917
I20260619 08:23:54.020947 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 31048
I20260619 08:23:54.020984 31186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:54.020836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:36539 (local address 127.0.0.1:39864)
0619 08:23:54.020908 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:54.020941 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.130:36539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 08:23:54.025485 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 30695
2026-06-19T08:23:54Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (661 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 08:23:54.045796 26110 mini_oidc.cc:152] Starting JWKS server
I20260619 08:23:54.046941 26110 webserver.cc:460] Webserver started at https://127.0.0.1:39061/ using document root <none> and password file <none>
I20260619 08:23:54.047039 26110 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 08:23:54.047199 26110 webserver.cc:460] Webserver started at http://127.0.0.1:41419/ using document root <none> and password file <none>
2026-06-19T08:23:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:23:54Z Disabled control of system clock
I20260619 08:23:54.057099 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:38869
--webserver_interface=127.25.127.190
--webserver_port=0
--builtin_ntp_servers=127.25.127.148:42603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:38869
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39061/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/testchainca.cert with env {}
W20260619 08:23:54.143965 31222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:54.144236 31222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:54.144276 31222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:54.145488 31222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:23:54.145555 31222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:54.145577 31222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:54.145599 31222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:23:54.145625 31222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 08:23:54.145656 31222 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 08:23:54.145679 31222 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39061/jwks.json
I20260619 08:23:54.147240 31222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:38869
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:38869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39061/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:54.147526 31222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:54.147765 31222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:54.152689 31230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:54.152690 31228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:54.152988 31222 server_base.cc:1034] running on GCE node
W20260619 08:23:54.153029 31227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:54.153333 31222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:54.153690 31222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:54.154858 31222 hybrid_clock.cc:648] HybridClock initialized: now 1781857434154842 us; error 29 us; skew 500 ppm
I20260619 08:23:54.156308 31222 webserver.cc:460] Webserver started at http://127.25.127.190:33903/ using document root <none> and password file <none>
I20260619 08:23:54.156563 31222 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:54.156639 31222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:54.156776 31222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:54.157873 31222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "97470558f33041a9ac5f3bb5ac586413"
format_stamp: "Formatted at 2026-06-19 08:23:54 on dist-test-slave-5rhg"
server_key: "6a012287768189c53b5ed73e0661262a"
server_key_iv: "53477a75fb99da23c94e528f124418b8"
server_key_version: "encryptionkey@0"
I20260619 08:23:54.158265 31222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "97470558f33041a9ac5f3bb5ac586413"
format_stamp: "Formatted at 2026-06-19 08:23:54 on dist-test-slave-5rhg"
server_key: "6a012287768189c53b5ed73e0661262a"
server_key_iv: "53477a75fb99da23c94e528f124418b8"
server_key_version: "encryptionkey@0"
I20260619 08:23:54.159588 31222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:23:54.160434 31236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:54.160611 31222 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:23:54.160719 31222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "97470558f33041a9ac5f3bb5ac586413"
format_stamp: "Formatted at 2026-06-19 08:23:54 on dist-test-slave-5rhg"
server_key: "6a012287768189c53b5ed73e0661262a"
server_key_iv: "53477a75fb99da23c94e528f124418b8"
server_key_version: "encryptionkey@0"
I20260619 08:23:54.160794 31222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:54.174835 31222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:54.175174 31222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:54.175326 31222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:54.180310 31222 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:38869
I20260619 08:23:54.180400 31288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:38869 every 8 connection(s)
I20260619 08:23:54.180789 31222 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:23:54.181795 31289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:54.184118 31289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413: Bootstrap starting.
I20260619 08:23:54.184773 31289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:54.185070 31289 log.cc:824] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:54.185334 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 31222
I20260619 08:23:54.185460 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:23:54.185672 26110 external_mini_cluster.cc:1428] Setting key 402b08ad5caba3ef1174fd142c4b0c00
I20260619 08:23:54.185882 31289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413: No bootstrap required, opened a new log
I20260619 08:23:54.188383 31289 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97470558f33041a9ac5f3bb5ac586413" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38869 } }
I20260619 08:23:54.188547 31289 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:54.188601 31289 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97470558f33041a9ac5f3bb5ac586413, State: Initialized, Role: FOLLOWER
I20260619 08:23:54.188745 31289 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97470558f33041a9ac5f3bb5ac586413" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38869 } }
I20260619 08:23:54.188833 31289 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:23:54.188864 31289 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:23:54.188906 31289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:54.189625 31289 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97470558f33041a9ac5f3bb5ac586413" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38869 } }
I20260619 08:23:54.189738 31289 leader_election.cc:302] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97470558f33041a9ac5f3bb5ac586413; no voters:
I20260619 08:23:54.190014 31289 leader_election.cc:288] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:23:54.190038 31294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:54.190241 31294 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413 [term 1 LEADER]: Becoming Leader. State: Replica: 97470558f33041a9ac5f3bb5ac586413, State: Running, Role: LEADER
I20260619 08:23:54.190341 31294 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97470558f33041a9ac5f3bb5ac586413" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38869 } }
I20260619 08:23:54.190416 31289 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:54.190726 31296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97470558f33041a9ac5f3bb5ac586413. Latest consensus state: current_term: 1 leader_uuid: "97470558f33041a9ac5f3bb5ac586413" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97470558f33041a9ac5f3bb5ac586413" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38869 } } }
I20260619 08:23:54.190984 31296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413 [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:54.190915 31295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97470558f33041a9ac5f3bb5ac586413" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97470558f33041a9ac5f3bb5ac586413" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38869 } } }
I20260619 08:23:54.191295 31295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413 [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:54.191316 31303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:23:54.191944 31303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:23:54.192286 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:54.186140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49702 (local address 127.25.127.190:38869)
0619 08:23:54.186378 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:54.186386 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:54.186411 (+ 25us) server_negotiation.cc:407] Connection header received
0619 08:23:54.187153 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:54.187161 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:54.187238 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:54.187388 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:54.187595 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.188531 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.189783 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.190333 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.190812 (+ 479us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:54.191254 (+ 442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:54.191258 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:54.191259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:23:54.191470 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:54.191520 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:23:54.191538 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:54.191920 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:54.191957 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 08:23:54.192042 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":79,"threads_started":1}
I20260619 08:23:54.193475 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:23:54.194005 31303 catalog_manager.cc:1347] Generated new cluster ID: a8c3cb1908574a11af2baa7a3a67ff6a
I20260619 08:23:54.194053 31303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:23:54.231113 31303 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:23:54.231910 31303 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:23:54.239769 31303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97470558f33041a9ac5f3bb5ac586413: Generated new TSK 0
I20260619 08:23:54.239977 31303 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 08:23:54.274744 31319 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 08:23:54.274881 31319 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 08:23:54.275156 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:54.252986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49714 (local address 127.25.127.190:38869)
0619 08:23:54.253054 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:54.253056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:54.253207 (+ 151us) server_negotiation.cc:407] Connection header received
0619 08:23:54.253315 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:54.253316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:54.253328 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:54.253388 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0619 08:23:54.253571 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.254038 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.255190 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.255383 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.255442 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:54.255703 (+ 261us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 08:23:54.274693 (+ 18990us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39061/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 08:23:54.275087 (+ 394us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49714: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39061/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":39}
W20260619 08:23:54.275220 31293 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49714: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39061/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260619 08:23:54.275312 31318 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:23:54.252923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:38869 (local address 127.0.0.1:49714)
0619 08:23:54.253129 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:54.253142 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:54.253218 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:54.253417 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:54.253418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:54.253420 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 08:23:54.253493 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:54.253495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.254127 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.254128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:54.255089 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:54.255092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.255473 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.255474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:54.255539 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:54.255562 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 08:23:54.274852 (+ 19290us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39061/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 08:23:54.274977 (+ 125us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.127.190:38869: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39061/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":119,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.25.127.190:38869: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39061/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 08:23:54.275689 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 31222
2026-06-19T08:23:54Z chronyd exiting
I20260619 08:23:54.311446 26110 test_util.cc:175] -----------------------------------------------
I20260619 08:23:54.311537 26110 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781857412505079-26110-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (266 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 08:23:54.312443 26110 mini_oidc.cc:152] Starting JWKS server
I20260619 08:23:54.313155 26110 webserver.cc:460] Webserver started at https://127.0.0.1:42097/ using document root <none> and password file <none>
I20260619 08:23:54.313235 26110 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 08:23:54.313423 26110 webserver.cc:460] Webserver started at http://127.0.0.1:35939/ using document root <none> and password file <none>
2026-06-19T08:23:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:23:54Z Disabled control of system clock
I20260619 08:23:54.321499 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:41547
--webserver_interface=127.25.127.190
--webserver_port=0
--builtin_ntp_servers=127.25.127.148:45625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:41547
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42097/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/testchainca.cert with env {}
W20260619 08:23:54.403126 31325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:54.403342 31325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:54.403365 31325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:54.404515 31325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:23:54.404552 31325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:54.404565 31325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:54.404577 31325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:23:54.404590 31325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 08:23:54.404603 31325 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 08:23:54.404613 31325 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42097/jwks.json
I20260619 08:23:54.406140 31325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:45625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:41547
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:41547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42097/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:54.406343 31325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:54.406502 31325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:54.411442 31333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:54.411463 31331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:54.411458 31330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:54.411543 31325 server_base.cc:1034] running on GCE node
I20260619 08:23:54.411892 31325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:54.412269 31325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:54.413463 31325 hybrid_clock.cc:648] HybridClock initialized: now 1781857434413448 us; error 44 us; skew 500 ppm
I20260619 08:23:54.414719 31325 webserver.cc:460] Webserver started at http://127.25.127.190:38079/ using document root <none> and password file <none>
I20260619 08:23:54.414963 31325 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:54.415071 31325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:54.415230 31325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:54.416409 31325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "58ae793b344e473f8c84c124907d5586"
format_stamp: "Formatted at 2026-06-19 08:23:54 on dist-test-slave-5rhg"
server_key: "028dca565d8c1e5dc312a21edb918ff5"
server_key_iv: "dfbe0ae2569def37547992072574ed4d"
server_key_version: "encryptionkey@0"
I20260619 08:23:54.416929 31325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "58ae793b344e473f8c84c124907d5586"
format_stamp: "Formatted at 2026-06-19 08:23:54 on dist-test-slave-5rhg"
server_key: "028dca565d8c1e5dc312a21edb918ff5"
server_key_iv: "dfbe0ae2569def37547992072574ed4d"
server_key_version: "encryptionkey@0"
I20260619 08:23:54.418393 31325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:23:54.419121 31339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:54.419301 31325 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:54.419406 31325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "58ae793b344e473f8c84c124907d5586"
format_stamp: "Formatted at 2026-06-19 08:23:54 on dist-test-slave-5rhg"
server_key: "028dca565d8c1e5dc312a21edb918ff5"
server_key_iv: "dfbe0ae2569def37547992072574ed4d"
server_key_version: "encryptionkey@0"
I20260619 08:23:54.419481 31325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:54.438684 31325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:54.439059 31325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:54.439227 31325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:54.443403 31325 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:41547
I20260619 08:23:54.443440 31391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:41547 every 8 connection(s)
I20260619 08:23:54.443815 31325 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:23:54.444507 31392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:54.445822 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 31325
I20260619 08:23:54.445919 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:23:54.446094 26110 external_mini_cluster.cc:1428] Setting key 28a7e07c77a63477e9388834f1bba5df
I20260619 08:23:54.448407 31392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586: Bootstrap starting.
I20260619 08:23:54.449196 31392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:54.449517 31392 log.cc:824] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:54.450256 31392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586: No bootstrap required, opened a new log
I20260619 08:23:54.452663 31395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:54.446640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43272 (local address 127.25.127.190:41547)
0619 08:23:54.446862 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:54.446871 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:54.446904 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:23:54.447506 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:54.447515 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:54.447602 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:54.447757 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:54.447939 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.449339 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.450317 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.450922 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.451042 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:54.451824 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:54.451828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:54.451829 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:23:54.452033 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:54.452074 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:23:54.452090 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:54.452208 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:54.452299 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 08:23:54.452405 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260619 08:23:54.452915 31392 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58ae793b344e473f8c84c124907d5586" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41547 } }
I20260619 08:23:54.453088 31392 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:54.453123 31392 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58ae793b344e473f8c84c124907d5586, State: Initialized, Role: FOLLOWER
I20260619 08:23:54.453225 31392 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58ae793b344e473f8c84c124907d5586" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41547 } }
I20260619 08:23:54.453305 31392 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:23:54.453336 31392 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:23:54.453377 31392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:54.453692 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:23:54.454075 31392 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58ae793b344e473f8c84c124907d5586" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41547 } }
I20260619 08:23:54.454196 31392 leader_election.cc:302] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58ae793b344e473f8c84c124907d5586; no voters:
I20260619 08:23:54.454363 31392 leader_election.cc:288] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:23:54.454452 31397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:54.454614 31392 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:54.454604 31397 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586 [term 1 LEADER]: Becoming Leader. State: Replica: 58ae793b344e473f8c84c124907d5586, State: Running, Role: LEADER
I20260619 08:23:54.454753 31397 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58ae793b344e473f8c84c124907d5586" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41547 } }
I20260619 08:23:54.455104 31397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58ae793b344e473f8c84c124907d5586. Latest consensus state: current_term: 1 leader_uuid: "58ae793b344e473f8c84c124907d5586" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58ae793b344e473f8c84c124907d5586" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41547 } } }
I20260619 08:23:54.455127 31399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58ae793b344e473f8c84c124907d5586" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58ae793b344e473f8c84c124907d5586" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41547 } } }
I20260619 08:23:54.455195 31397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586 [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:54.455199 31399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586 [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:54.455405 31404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:23:54.455902 31404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:23:54.457587 31404 catalog_manager.cc:1347] Generated new cluster ID: e790fa6227314743ac60a51497cd1b58
I20260619 08:23:54.457638 31404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:23:54.477385 31404 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:23:54.477993 31404 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:23:54.484536 31404 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58ae793b344e473f8c84c124907d5586: Generated new TSK 0
I20260619 08:23:54.484756 31404 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 08:23:54.506057 31422 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 08:23:54.506156 31422 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 08:23:54.506505 31395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:54.491620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43278 (local address 127.25.127.190:41547)
0619 08:23:54.491668 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:54.491670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:54.491788 (+ 118us) server_negotiation.cc:407] Connection header received
0619 08:23:54.491869 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:54.491869 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:54.491880 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:54.491944 (+ 64us) server_negotiation.cc:226] Negotiated authn=JWT
0619 08:23:54.492261 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.492802 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.493946 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.494133 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.494191 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:54.494424 (+ 233us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 08:23:54.506313 (+ 11889us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 08:23:54.506449 (+ 136us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43278: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":25}
W20260619 08:23:54.506582 31395 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43278: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260619 08:23:54.506709 31421 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:23:54.491519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:41547 (local address 127.0.0.1:43278)
0619 08:23:54.491711 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:54.491724 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:54.491795 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:54.491958 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:54.491959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:54.491961 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 08:23:54.492025 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:54.492027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.492895 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.492897 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:54.493857 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:54.493862 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.494240 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.494242 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:54.494318 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:54.494325 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 08:23:54.506441 (+ 12116us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 08:23:54.506601 (+ 160us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.127.190:41547: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":110,"threads_started":1}
I20260619 08:23:54.507107 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 31325
2026-06-19T08:23:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (232 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 08:23:54.545267 26110 mini_oidc.cc:152] Starting JWKS server
I20260619 08:23:54.545943 26110 webserver.cc:460] Webserver started at https://127.0.0.1:42027/ using document root <none> and password file <none>
I20260619 08:23:54.545985 26110 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 08:23:54.546094 26110 webserver.cc:460] Webserver started at http://127.0.0.1:37759/ using document root <none> and password file <none>
2026-06-19T08:23:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:23:54Z Disabled control of system clock
I20260619 08:23:54.554282 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:39631
--webserver_interface=127.25.127.190
--webserver_port=0
--builtin_ntp_servers=127.25.127.148:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:39631
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42027/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 08:23:54.636010 31428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:54.636302 31428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:54.636332 31428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:54.637571 31428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:23:54.637611 31428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:54.637624 31428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:54.637638 31428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:23:54.637689 31428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 08:23:54.637714 31428 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 08:23:54.637740 31428 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42027/jwks.json
I20260619 08:23:54.639154 31428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:39631
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:39631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42027/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:54.639417 31428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:54.639730 31428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:54.645229 31433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:54.645236 31436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:54.645249 31434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:54.645601 31428 server_base.cc:1034] running on GCE node
I20260619 08:23:54.645875 31428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:54.646178 31428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:54.647326 31428 hybrid_clock.cc:648] HybridClock initialized: now 1781857434647324 us; error 38 us; skew 500 ppm
I20260619 08:23:54.653228 31428 webserver.cc:460] Webserver started at http://127.25.127.190:45053/ using document root <none> and password file <none>
I20260619 08:23:54.653524 31428 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:54.653575 31428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:54.653748 31428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:54.654883 31428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "267c708ef11e46cdbb051a93341d5a83"
format_stamp: "Formatted at 2026-06-19 08:23:54 on dist-test-slave-5rhg"
server_key: "7f918075c4a0723ebe82273e03bf78b7"
server_key_iv: "22a94848f9621025de7deab2efe180ee"
server_key_version: "encryptionkey@0"
I20260619 08:23:54.655259 31428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "267c708ef11e46cdbb051a93341d5a83"
format_stamp: "Formatted at 2026-06-19 08:23:54 on dist-test-slave-5rhg"
server_key: "7f918075c4a0723ebe82273e03bf78b7"
server_key_iv: "22a94848f9621025de7deab2efe180ee"
server_key_version: "encryptionkey@0"
I20260619 08:23:54.656755 31428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:23:54.657639 31442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:54.657862 31428 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:54.657951 31428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "267c708ef11e46cdbb051a93341d5a83"
format_stamp: "Formatted at 2026-06-19 08:23:54 on dist-test-slave-5rhg"
server_key: "7f918075c4a0723ebe82273e03bf78b7"
server_key_iv: "22a94848f9621025de7deab2efe180ee"
server_key_version: "encryptionkey@0"
I20260619 08:23:54.658010 31428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:54.688993 31428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:54.689294 31428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:54.689400 31428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:54.693270 31428 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:39631
I20260619 08:23:54.693326 31494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:39631 every 8 connection(s)
I20260619 08:23:54.693657 31428 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:23:54.694327 31495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:54.697082 31495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83: Bootstrap starting.
I20260619 08:23:54.697683 31495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:54.697974 31495 log.cc:824] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:54.698673 31495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83: No bootstrap required, opened a new log
I20260619 08:23:54.698845 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 31428
I20260619 08:23:54.698931 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:23:54.699115 26110 external_mini_cluster.cc:1428] Setting key 55bbaa5fee8a581494a80d142995529d
I20260619 08:23:54.700847 31495 raft_consensus.cc:348] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267c708ef11e46cdbb051a93341d5a83" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39631 } }
I20260619 08:23:54.701007 31495 raft_consensus.cc:374] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:54.701054 31495 raft_consensus.cc:729] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 267c708ef11e46cdbb051a93341d5a83, State: Initialized, Role: FOLLOWER
I20260619 08:23:54.701213 31495 consensus_queue.cc:260] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267c708ef11e46cdbb051a93341d5a83" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39631 } }
I20260619 08:23:54.701316 31495 raft_consensus.cc:388] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:23:54.701363 31495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:23:54.701416 31495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:54.702008 31495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267c708ef11e46cdbb051a93341d5a83" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39631 } }
I20260619 08:23:54.702131 31495 leader_election.cc:302] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 267c708ef11e46cdbb051a93341d5a83; no voters:
I20260619 08:23:54.702335 31495 leader_election.cc:288] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:23:54.702360 31500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:54.702523 31500 raft_consensus.cc:686] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83 [term 1 LEADER]: Becoming Leader. State: Replica: 267c708ef11e46cdbb051a93341d5a83, State: Running, Role: LEADER
I20260619 08:23:54.702605 31500 consensus_queue.cc:237] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267c708ef11e46cdbb051a93341d5a83" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39631 } }
I20260619 08:23:54.702613 31495 sys_catalog.cc:564] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:54.702924 31500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 267c708ef11e46cdbb051a93341d5a83. Latest consensus state: current_term: 1 leader_uuid: "267c708ef11e46cdbb051a93341d5a83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267c708ef11e46cdbb051a93341d5a83" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39631 } } }
I20260619 08:23:54.702996 31500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83 [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:54.702946 31502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "267c708ef11e46cdbb051a93341d5a83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267c708ef11e46cdbb051a93341d5a83" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39631 } } }
I20260619 08:23:54.703141 31502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83 [sys.catalog]: This master's current role is: LEADER
W20260619 08:23:54.704370 31515 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:23:54.704454 31515 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:23:54.704674 31505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:23:54.704840 31505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:23:54.705551 31499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:54.699661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45668 (local address 127.25.127.190:39631)
0619 08:23:54.699893 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:54.699901 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:54.699926 (+ 25us) server_negotiation.cc:407] Connection header received
0619 08:23:54.700499 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:54.700506 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:54.700567 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:54.700725 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:54.700904 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.702321 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.703387 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.703879 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.704016 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:54.704874 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:54.704878 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:54.704879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:23:54.705057 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:54.705092 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:23:54.705102 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:54.705174 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:54.705256 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 08:23:54.705338 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":85,"threads_started":1}
I20260619 08:23:54.706450 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:23:54.706604 31505 catalog_manager.cc:1347] Generated new cluster ID: 01b9f43e6ebb4211837626912f36742d
I20260619 08:23:54.706660 31505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:23:54.738310 31505 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:23:54.739030 31505 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:23:54.752138 31505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 267c708ef11e46cdbb051a93341d5a83: Generated new TSK 0
I20260619 08:23:54.752415 31505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:23:54.763373 31499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:54.759515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45684 (local address 127.25.127.190:39631)
0619 08:23:54.759563 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:54.759565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:54.759638 (+ 73us) server_negotiation.cc:407] Connection header received
0619 08:23:54.759703 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:54.759704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:54.759715 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:54.759811 (+ 96us) server_negotiation.cc:226] Negotiated authn=JWT
0619 08:23:54.759923 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.760720 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.761567 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.761882 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.761951 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:54.762134 (+ 183us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 08:23:54.763137 (+ 1003us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42027/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 08:23:54.763278 (+ 141us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45684: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42027/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":26}
W20260619 08:23:54.763442 31499 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45684: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42027/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 08:23:54.765311 31524 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:23:54.759408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:39631 (local address 127.0.0.1:45684)
0619 08:23:54.759561 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:54.759571 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:54.759639 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:54.759808 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:54.759808 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:54.759810 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 08:23:54.759857 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:54.759859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.760840 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.760842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:54.761472 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:54.761474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.761959 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.761960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:54.762009 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:54.762015 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 08:23:54.763246 (+ 1231us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42027/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 08:23:54.765212 (+ 1966us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.127.190:39631: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42027/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":1720,"thread_start_us":108,"threads_started":1}
I20260619 08:23:54.765604 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 31428
E20260619 08:23:54.774385 31525 webserver.cc:533] Webserver: sslize failed:
E20260619 08:23:54.774461 31525 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-19T08:23:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (258 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T08:23:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:23:54Z Disabled control of system clock
I20260619 08:23:54.811245 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:39295
--webserver_interface=127.25.127.190
--webserver_port=0
--builtin_ntp_servers=127.25.127.148:37085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:39295
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/testchainca.cert with env {}
W20260619 08:23:54.898142 31529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:54.898391 31529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:54.898434 31529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:54.899657 31529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:23:54.899724 31529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:54.899744 31529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:54.899765 31529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:23:54.899796 31529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 08:23:54.899818 31529 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 08:23:54.899847 31529 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 08:23:54.901367 31529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:37085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:39295
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:39295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:54.901659 31529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:54.901882 31529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:54.906937 31537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:54.906997 31534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:54.906936 31535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:54.907104 31529 server_base.cc:1034] running on GCE node
I20260619 08:23:54.907411 31529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:54.907739 31529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:54.908913 31529 hybrid_clock.cc:648] HybridClock initialized: now 1781857434908902 us; error 32 us; skew 500 ppm
I20260619 08:23:54.910195 31529 webserver.cc:460] Webserver started at http://127.25.127.190:44767/ using document root <none> and password file <none>
I20260619 08:23:54.910480 31529 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:54.910559 31529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:54.910714 31529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:54.911867 31529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "a0285bb1099344a5a539b989d34fc37a"
format_stamp: "Formatted at 2026-06-19 08:23:54 on dist-test-slave-5rhg"
server_key: "4f8b7b3700e3a8b01c74190fcfcb0f90"
server_key_iv: "782f6fadedb545b50c12ea1ef1776d7a"
server_key_version: "encryptionkey@0"
I20260619 08:23:54.912331 31529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "a0285bb1099344a5a539b989d34fc37a"
format_stamp: "Formatted at 2026-06-19 08:23:54 on dist-test-slave-5rhg"
server_key: "4f8b7b3700e3a8b01c74190fcfcb0f90"
server_key_iv: "782f6fadedb545b50c12ea1ef1776d7a"
server_key_version: "encryptionkey@0"
I20260619 08:23:54.913770 31529 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:54.914654 31543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:54.914883 31529 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:54.914994 31529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "a0285bb1099344a5a539b989d34fc37a"
format_stamp: "Formatted at 2026-06-19 08:23:54 on dist-test-slave-5rhg"
server_key: "4f8b7b3700e3a8b01c74190fcfcb0f90"
server_key_iv: "782f6fadedb545b50c12ea1ef1776d7a"
server_key_version: "encryptionkey@0"
I20260619 08:23:54.915072 31529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:54.964687 31529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:54.965049 31529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:54.965219 31529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:54.969399 31529 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:39295
I20260619 08:23:54.969437 31595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:39295 every 8 connection(s)
I20260619 08:23:54.969806 31529 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:23:54.970849 31596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:54.973224 31596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a: Bootstrap starting.
I20260619 08:23:54.973841 31596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:54.974159 31596 log.cc:824] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:54.974882 31596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a: No bootstrap required, opened a new log
I20260619 08:23:54.976619 31596 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0285bb1099344a5a539b989d34fc37a" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39295 } }
I20260619 08:23:54.976771 31596 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:54.976794 31596 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0285bb1099344a5a539b989d34fc37a, State: Initialized, Role: FOLLOWER
I20260619 08:23:54.976956 31596 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0285bb1099344a5a539b989d34fc37a" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39295 } }
I20260619 08:23:54.977022 31596 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:23:54.977044 31596 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:23:54.977108 31596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:54.977694 31596 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0285bb1099344a5a539b989d34fc37a" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39295 } }
I20260619 08:23:54.977782 31596 leader_election.cc:302] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0285bb1099344a5a539b989d34fc37a; no voters:
I20260619 08:23:54.977977 31596 leader_election.cc:288] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:23:54.978013 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 31529
I20260619 08:23:54.978104 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:23:54.978135 31599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:54.978271 31596 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:54.978312 26110 external_mini_cluster.cc:1428] Setting key 65a1511d2ac9829a365e3325e5e125ba
I20260619 08:23:54.978372 31599 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a [term 1 LEADER]: Becoming Leader. State: Replica: a0285bb1099344a5a539b989d34fc37a, State: Running, Role: LEADER
I20260619 08:23:54.978488 31599 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0285bb1099344a5a539b989d34fc37a" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39295 } }
I20260619 08:23:54.979588 31601 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0285bb1099344a5a539b989d34fc37a. Latest consensus state: current_term: 1 leader_uuid: "a0285bb1099344a5a539b989d34fc37a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0285bb1099344a5a539b989d34fc37a" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39295 } } }
I20260619 08:23:54.979681 31601 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:54.979766 31600 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0285bb1099344a5a539b989d34fc37a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0285bb1099344a5a539b989d34fc37a" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39295 } } }
I20260619 08:23:54.980058 31600 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a [sys.catalog]: This master's current role is: LEADER
W20260619 08:23:54.980669 31615 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:23:54.980746 31615 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:23:54.981316 31616 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:23:54.981494 31616 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:23:54.983107 31616 catalog_manager.cc:1347] Generated new cluster ID: a9e1118d7af94323be2e34aa469f68d1
I20260619 08:23:54.983160 31616 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:23:54.984949 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:54.978957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37628 (local address 127.25.127.190:39295)
0619 08:23:54.979190 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:54.979200 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:54.979237 (+ 37us) server_negotiation.cc:407] Connection header received
0619 08:23:54.979348 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:54.979353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:54.979445 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:54.979875 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:54.979982 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.981666 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.982636 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:54.983242 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:54.983371 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:54.984129 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:54.984133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:54.984134 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:23:54.984417 (+ 283us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:54.984463 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:23:54.984479 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:54.984568 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:54.984642 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 08:23:54.984728 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":70,"threads_started":1}
I20260619 08:23:54.989239 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:23:55.007628 31616 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:23:55.008414 31616 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:23:55.011957 31616 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0285bb1099344a5a539b989d34fc37a: Generated new TSK 0
I20260619 08:23:55.012187 31616 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:23:55.091480 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:55.020735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37632 (local address 127.25.127.190:39295)
0619 08:23:55.020797 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:55.020799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:55.020916 (+ 117us) server_negotiation.cc:407] Connection header received
0619 08:23:55.020999 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:55.021000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:55.021011 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:55.021075 (+ 64us) server_negotiation.cc:226] Negotiated authn=JWT
0619 08:23:55.021231 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:55.021732 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:55.022889 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:55.023076 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:55.023128 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:55.023337 (+ 209us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 08:23:55.091265 (+ 67928us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 08:23:55.091422 (+ 157us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37632: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":35}
W20260619 08:23:55.091581 31608 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37632: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 08:23:55.091681 31625 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:23:55.020652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:39295 (local address 127.0.0.1:37632)
0619 08:23:55.020844 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:55.020856 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:55.020923 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:55.021093 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:55.021093 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:55.021095 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 08:23:55.021162 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:55.021165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:55.021841 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:55.021842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:55.022801 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:55.022804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:55.023162 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:55.023163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:55.023227 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:55.023234 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 08:23:55.091419 (+ 68185us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 08:23:55.091571 (+ 152us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.127.190:39295: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":100,"threads_started":1}
I20260619 08:23:55.092068 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 31529
2026-06-19T08:23:55Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (306 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (89 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (89 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:23:55 dist-test-slave-5rhg krb5kdc[31637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:23:55 dist-test-slave-5rhg krb5kdc[31637](info): set up 2 sockets
Jun 19 08:23:55 dist-test-slave-5rhg krb5kdc[31637](info): commencing operation
krb5kdc: starting...
W20260619 08:23:56.321159 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:23:56 dist-test-slave-5rhg krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:23:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:23:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:23:56.438364 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:41383
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:46429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:41383
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:56.520360 31650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:56.520570 31650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:56.520592 31650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:56.521787 31650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:23:56.521829 31650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:56.521843 31650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:56.521855 31650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:23:56.521867 31650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:23:56.523413 31650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:46429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:41383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:41383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:56.523643 31650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:56.523862 31650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:56.528841 31655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:56.528865 31656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:56.528844 31658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:56.529178 31650 server_base.cc:1034] running on GCE node
I20260619 08:23:56.529386 31650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:56.529685 31650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:56.530853 31650 hybrid_clock.cc:648] HybridClock initialized: now 1781857436530841 us; error 40 us; skew 500 ppm
Jun 19 08:23:56 dist-test-slave-5rhg krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857436, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:56.533346 31650 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:23:56.533810 31650 webserver.cc:460] Webserver started at http://127.25.127.190:40689/ using document root <none> and password file <none>
I20260619 08:23:56.534046 31650 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:56.534091 31650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:56.534241 31650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:56.535332 31650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "f7e030a783034afdb81039f58a02c15b"
format_stamp: "Formatted at 2026-06-19 08:23:56 on dist-test-slave-5rhg"
server_key: "14b2f9ab6700b6921b5447c92d92ac48"
server_key_iv: "6e9f5faee50c91d38009f5c5fa2f4f8b"
server_key_version: "encryptionkey@0"
I20260619 08:23:56.535704 31650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "f7e030a783034afdb81039f58a02c15b"
format_stamp: "Formatted at 2026-06-19 08:23:56 on dist-test-slave-5rhg"
server_key: "14b2f9ab6700b6921b5447c92d92ac48"
server_key_iv: "6e9f5faee50c91d38009f5c5fa2f4f8b"
server_key_version: "encryptionkey@0"
I20260619 08:23:56.537206 31650 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:23:56.537935 31665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:56.538106 31650 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:56.538193 31650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "f7e030a783034afdb81039f58a02c15b"
format_stamp: "Formatted at 2026-06-19 08:23:56 on dist-test-slave-5rhg"
server_key: "14b2f9ab6700b6921b5447c92d92ac48"
server_key_iv: "6e9f5faee50c91d38009f5c5fa2f4f8b"
server_key_version: "encryptionkey@0"
I20260619 08:23:56.538249 31650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:56.546890 31650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:56.549203 31650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:56.549314 31650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:56.553117 31650 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:41383
I20260619 08:23:56.553154 31717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:41383 every 8 connection(s)
I20260619 08:23:56.553509 31650 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:23:56.554383 31718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:56.556715 31718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b: Bootstrap starting.
I20260619 08:23:56.557235 31718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:56.557475 31718 log.cc:824] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:56.558102 31718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b: No bootstrap required, opened a new log
I20260619 08:23:56.559659 31718 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e030a783034afdb81039f58a02c15b" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41383 } }
I20260619 08:23:56.559790 31718 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:56.559813 31718 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e030a783034afdb81039f58a02c15b, State: Initialized, Role: FOLLOWER
I20260619 08:23:56.559947 31718 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e030a783034afdb81039f58a02c15b" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41383 } }
I20260619 08:23:56.560029 31718 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:23:56.560053 31718 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:23:56.560081 31718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:56.560624 31718 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e030a783034afdb81039f58a02c15b" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41383 } }
I20260619 08:23:56.560710 31718 leader_election.cc:302] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7e030a783034afdb81039f58a02c15b; no voters:
I20260619 08:23:56.560837 31718 leader_election.cc:288] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:23:56.560936 31721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:56.561143 31718 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:56.561331 31721 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b [term 1 LEADER]: Becoming Leader. State: Replica: f7e030a783034afdb81039f58a02c15b, State: Running, Role: LEADER
I20260619 08:23:56.561443 31721 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e030a783034afdb81039f58a02c15b" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41383 } }
I20260619 08:23:56.561890 31722 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7e030a783034afdb81039f58a02c15b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e030a783034afdb81039f58a02c15b" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41383 } } }
I20260619 08:23:56.561901 31723 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7e030a783034afdb81039f58a02c15b. Latest consensus state: current_term: 1 leader_uuid: "f7e030a783034afdb81039f58a02c15b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e030a783034afdb81039f58a02c15b" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41383 } } }
I20260619 08:23:56.561990 31722 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:56.562024 31723 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:56.562386 31730 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:23:56.562896 31730 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:23:56.563625 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 31650
I20260619 08:23:56.563740 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:23:56.563917 26110 external_mini_cluster.cc:1428] Setting key 3e98d3814d2a9cb8317e6de307b88662
I20260619 08:23:56.564710 31730 catalog_manager.cc:1347] Generated new cluster ID: b77b1ee811ff491c808f15ab027c3e48
I20260619 08:23:56.564762 31730 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:23:56 dist-test-slave-5rhg krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:56.576030 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:56.564573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41164 (local address 127.25.127.190:41383)
0619 08:23:56.564792 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:56.564801 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:56.564833 (+ 32us) server_negotiation.cc:407] Connection header received
0619 08:23:56.564953 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:56.564960 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:56.565013 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:56.565147 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:56.565952 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:56.567362 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:56.568503 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:56.569117 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:56.569211 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:56.572173 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:56.572177 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:56.572178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:56.572191 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:56.574225 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:56.574620 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:56.574623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:56.574623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:56.574737 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:56.574942 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:56.574942 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:56.574943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:56.575298 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:56.575457 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:56.575527 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 08:23:56.575783 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":148,"threads_started":1}
I20260619 08:23:56.584653 31730 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:23:56.585379 31730 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:23:56.596340 31730 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7e030a783034afdb81039f58a02c15b: Generated new TSK 0
I20260619 08:23:56.596570 31730 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
I20260619 08:23:56.624483 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:41383
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:46429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:56.708072 31744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:56.708340 31744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:56.708366 31744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:56.709497 31744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:56.709530 31744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:56.709556 31744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:23:56.710994 31744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:46429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:41383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:56.711194 31744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:56.711370 31744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:56.716526 31749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:56.716503 31752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:56.716507 31750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:56.716771 31744 server_base.cc:1034] running on GCE node
I20260619 08:23:56.717005 31744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:56.717259 31744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:56.718464 31744 hybrid_clock.cc:648] HybridClock initialized: now 1781857436718439 us; error 37 us; skew 500 ppm
Jun 19 08:23:56 dist-test-slave-5rhg krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857436, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:56.721199 31744 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:23:56.721644 31744 webserver.cc:460] Webserver started at http://127.25.127.129:34559/ using document root <none> and password file <none>
I20260619 08:23:56.721875 31744 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:56.721944 31744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:56.722097 31744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:56.723163 31744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "e93d70319748432e870f8db5916229d5"
format_stamp: "Formatted at 2026-06-19 08:23:56 on dist-test-slave-5rhg"
server_key: "80b5ed959804b646ce99a5cdf4cb91a6"
server_key_iv: "f2bf318abd63dd7a6094d35b225a6098"
server_key_version: "encryptionkey@0"
I20260619 08:23:56.723527 31744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "e93d70319748432e870f8db5916229d5"
format_stamp: "Formatted at 2026-06-19 08:23:56 on dist-test-slave-5rhg"
server_key: "80b5ed959804b646ce99a5cdf4cb91a6"
server_key_iv: "f2bf318abd63dd7a6094d35b225a6098"
server_key_version: "encryptionkey@0"
I20260619 08:23:56.724947 31744 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:23:56.725719 31759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:56.725884 31744 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:23:56.725991 31744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "e93d70319748432e870f8db5916229d5"
format_stamp: "Formatted at 2026-06-19 08:23:56 on dist-test-slave-5rhg"
server_key: "80b5ed959804b646ce99a5cdf4cb91a6"
server_key_iv: "f2bf318abd63dd7a6094d35b225a6098"
server_key_version: "encryptionkey@0"
I20260619 08:23:56.726068 31744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:56.738338 31744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:56.740970 31744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:56.741137 31744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:56.741379 31744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:56.741717 31744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:56.741773 31744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:56.741814 31744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:56.741842 31744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:56.747946 31744 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:38009
I20260619 08:23:56.748018 31872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:38009 every 8 connection(s)
I20260619 08:23:56.748373 31744 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:23:56.749327 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 31744
I20260619 08:23:56.749451 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:23:56.749642 26110 external_mini_cluster.cc:1428] Setting key aa9fc7bfb22e9c6ce4b38fe7dee1bb8c
Jun 19 08:23:56 dist-test-slave-5rhg krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857436, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.127.130@KRBTEST.COM; defaulting to no policy
I20260619 08:23:56.758982 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:56.749113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:56175 (local address 127.25.127.190:41383)
0619 08:23:56.749190 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:56.749192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:56.749374 (+ 182us) server_negotiation.cc:407] Connection header received
0619 08:23:56.750405 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:56.750406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:56.750421 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:56.750494 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:56.751130 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:56.751863 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:56.753204 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:56.753566 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:56.753666 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:56.755833 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:56.755834 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:56.755835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:56.755848 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:56.757615 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:56.758015 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:56.758017 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:56.758017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:56.758077 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:56.758324 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:56.758325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:56.758325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:56.758492 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:56.758576 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:56.758839 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 08:23:56.758914 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 08:23:56.759135 31875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:56.749066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:41383 (local address 127.25.127.129:56175)
0619 08:23:56.749237 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:56.749279 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:56.750322 (+ 1043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:56.750551 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:56.750554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:56.750828 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:56.751024 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:56.751028 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:56.751969 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:56.751971 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:56.753111 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:56.753113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:56.753690 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:56.753691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:56.753799 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:56.754356 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:56.754364 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:56.755734 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:56.757704 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:56.757711 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:56.757712 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:56.757934 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:56.758156 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:56.758156 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:56.758156 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:56.758258 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:56.758566 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:56.758571 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:56.758697 (+ 126us) client_negotiation.cc:769] Sending connection context
0619 08:23:56.758785 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 08:23:56.758914 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":103,"threads_started":1}
I20260619 08:23:56.759683 31873 heartbeater.cc:347] Connected to a master server at 127.25.127.190:41383
I20260619 08:23:56.759773 31873 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:56.759912 31873 heartbeater.cc:511] Master 127.25.127.190:41383 requested a full tablet report, sending...
I20260619 08:23:56.760495 31682 ts_manager.cc:194] Registered new tserver with Master: e93d70319748432e870f8db5916229d5 (127.25.127.129:38009)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.130@KRBTEST.COM" created.
I20260619 08:23:56.761415 31682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:56175
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
I20260619 08:23:56.775338 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:41383
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:46429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:56.857184 31878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:56.857441 31878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:56.857481 31878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:56.858696 31878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:56.858773 31878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:56.858821 31878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:23:56.860498 31878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:46429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:41383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:56.860801 31878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:56.861048 31878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:56.866686 31884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:56.866649 31883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:56.866688 31878 server_base.cc:1034] running on GCE node
W20260619 08:23:56.866667 31886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:56.867134 31878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:56.867362 31878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:56.868489 31878 hybrid_clock.cc:648] HybridClock initialized: now 1781857436868464 us; error 45 us; skew 500 ppm
Jun 19 08:23:56 dist-test-slave-5rhg krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857436, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:56.871078 31878 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:23:56.871546 31878 webserver.cc:460] Webserver started at http://127.25.127.130:40363/ using document root <none> and password file <none>
I20260619 08:23:56.871763 31878 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:56.871804 31878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:56.871896 31878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:56.872992 31878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "4723e11edf3c4110a0cce61466a117ec"
format_stamp: "Formatted at 2026-06-19 08:23:56 on dist-test-slave-5rhg"
server_key: "11e2da204ce50b51f0b20ade162894c0"
server_key_iv: "d2d160beeb913f038d43886d91a35988"
server_key_version: "encryptionkey@0"
I20260619 08:23:56.873307 31878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "4723e11edf3c4110a0cce61466a117ec"
format_stamp: "Formatted at 2026-06-19 08:23:56 on dist-test-slave-5rhg"
server_key: "11e2da204ce50b51f0b20ade162894c0"
server_key_iv: "d2d160beeb913f038d43886d91a35988"
server_key_version: "encryptionkey@0"
I20260619 08:23:56.874636 31878 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:23:56.875392 31893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:56.875649 31878 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:56.875753 31878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "4723e11edf3c4110a0cce61466a117ec"
format_stamp: "Formatted at 2026-06-19 08:23:56 on dist-test-slave-5rhg"
server_key: "11e2da204ce50b51f0b20ade162894c0"
server_key_iv: "d2d160beeb913f038d43886d91a35988"
server_key_version: "encryptionkey@0"
I20260619 08:23:56.875825 31878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:56.899271 31878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:56.901870 31878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:56.901993 31878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:56.902213 31878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:56.902565 31878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:56.902603 31878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:56.902650 31878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:56.902680 31878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:56.908622 31878 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:33417
I20260619 08:23:56.908721 32006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:33417 every 8 connection(s)
I20260619 08:23:56.909034 31878 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:23:56.910082 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 31878
I20260619 08:23:56.910208 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:23:56.910419 26110 external_mini_cluster.cc:1428] Setting key 3bc8f00a66cf217bda9820f43c02beea
Jun 19 08:23:56 dist-test-slave-5rhg krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857436, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.127.131@KRBTEST.COM; defaulting to no policy
I20260619 08:23:56.920027 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:56.909967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:60591 (local address 127.25.127.190:41383)
0619 08:23:56.910059 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:56.910062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:56.910233 (+ 171us) server_negotiation.cc:407] Connection header received
0619 08:23:56.911281 (+ 1048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:56.911283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:56.911294 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:56.911414 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:56.912196 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:56.913004 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:56.914030 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:56.914379 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:56.914461 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:56.917152 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:56.917154 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:56.917154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:56.917161 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:56.918589 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:56.919091 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:56.919092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:56.919092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:56.919150 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:56.919377 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:56.919377 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:56.919377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:56.919477 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:56.919526 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:56.919821 (+ 295us) server_negotiation.cc:299] Negotiation successful
0619 08:23:56.919916 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 08:23:56.920130 32009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:56.909918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:41383 (local address 127.25.127.130:60591)
0619 08:23:56.910077 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:56.910121 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:56.911167 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:56.911517 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:56.911522 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:56.911831 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:56.912057 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:56.912062 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:56.913115 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:56.913116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:56.913932 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:56.913934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:56.914521 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:56.914523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:56.914678 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:56.915269 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:56.915277 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:56.917051 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:56.918683 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:56.918690 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:56.918692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:56.919006 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:56.919223 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:56.919224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:56.919224 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:56.919325 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:56.919528 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:56.919533 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:56.919665 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 08:23:56.919754 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 08:23:56.919931 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260619 08:23:56.920586 32007 heartbeater.cc:347] Connected to a master server at 127.25.127.190:41383
I20260619 08:23:56.920674 32007 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:56.920882 32007 heartbeater.cc:511] Master 127.25.127.190:41383 requested a full tablet report, sending...
I20260619 08:23:56.921185 31682 ts_manager.cc:194] Registered new tserver with Master: 4723e11edf3c4110a0cce61466a117ec (127.25.127.130:33417)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.131@KRBTEST.COM" created.
I20260619 08:23:56.921631 31682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:60591
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
I20260619 08:23:56.935222 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:41383
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:46429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:57.016748 32012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:57.017006 32012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:57.017030 32012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:57.018234 32012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:57.018277 32012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:57.018304 32012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:23:57.019815 32012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:46429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:41383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:57.020063 32012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:57.020327 32012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:57.025280 32017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:57.025502 32012 server_base.cc:1034] running on GCE node
W20260619 08:23:57.025403 32018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:57.025280 32020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:57.025867 32012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:57.026121 32012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:57.027293 32012 hybrid_clock.cc:648] HybridClock initialized: now 1781857437027287 us; error 33 us; skew 500 ppm
Jun 19 08:23:57 dist-test-slave-5rhg krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857437, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:57.030038 32012 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:23:57.030440 32012 webserver.cc:460] Webserver started at http://127.25.127.131:33937/ using document root <none> and password file <none>
I20260619 08:23:57.030669 32012 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:57.030751 32012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:57.030911 32012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:57.031956 32012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "2f0450a6d309480d94c8614481d97755"
format_stamp: "Formatted at 2026-06-19 08:23:57 on dist-test-slave-5rhg"
server_key: "46a24ad676d82420fa745d57a451ce77"
server_key_iv: "9026e558068478e52d7b75caaf1d869d"
server_key_version: "encryptionkey@0"
I20260619 08:23:57.032356 32012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "2f0450a6d309480d94c8614481d97755"
format_stamp: "Formatted at 2026-06-19 08:23:57 on dist-test-slave-5rhg"
server_key: "46a24ad676d82420fa745d57a451ce77"
server_key_iv: "9026e558068478e52d7b75caaf1d869d"
server_key_version: "encryptionkey@0"
I20260619 08:23:57.033685 32012 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:23:57.034488 32027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:57.034688 32012 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:23:57.034792 32012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "2f0450a6d309480d94c8614481d97755"
format_stamp: "Formatted at 2026-06-19 08:23:57 on dist-test-slave-5rhg"
server_key: "46a24ad676d82420fa745d57a451ce77"
server_key_iv: "9026e558068478e52d7b75caaf1d869d"
server_key_version: "encryptionkey@0"
I20260619 08:23:57.034869 32012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:57.050578 32012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:57.053349 32012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:57.053532 32012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:57.053794 32012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:57.054133 32012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:57.054195 32012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:57.054229 32012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:57.054258 32012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:57.059971 32012 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:36403
I20260619 08:23:57.060055 32140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:36403 every 8 connection(s)
I20260619 08:23:57.060406 32012 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:23:57 dist-test-slave-5rhg krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857437, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:23:57.070328 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 32012
I20260619 08:23:57.070422 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:23:57.070457 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:57.061305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:47019 (local address 127.25.127.190:41383)
0619 08:23:57.061378 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:57.061381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:57.061597 (+ 216us) server_negotiation.cc:407] Connection header received
0619 08:23:57.062873 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:57.062875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:57.062888 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:57.062955 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:57.063575 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:57.064066 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:57.065123 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:57.065316 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:57.065369 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:57.067678 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:57.067679 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:57.067680 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:57.067685 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:57.069141 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:57.069516 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:57.069517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:57.069517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:57.069565 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:57.069780 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:57.069780 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:57.069780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:57.069882 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:57.069929 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:57.070254 (+ 325us) server_negotiation.cc:299] Negotiation successful
0619 08:23:57.070346 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 08:23:57.070621 26110 external_mini_cluster.cc:1428] Setting key 6c8860fc5cf20e0ad05e777d8e7be45d
I20260619 08:23:57.070669 32143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:57.061204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:41383 (local address 127.25.127.131:47019)
0619 08:23:57.061467 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:57.061507 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:57.062744 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:57.063035 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:57.063038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:57.063301 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:57.063500 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:57.063504 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:57.064205 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:57.064206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:57.065048 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:57.065050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:57.065390 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:57.065391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:57.065493 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:57.066123 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:57.066132 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:57.067596 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:57.069218 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:57.069224 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:57.069226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:57.069450 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:57.069624 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:57.069624 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:57.069625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:57.069727 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:57.069971 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:57.069976 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:57.070116 (+ 140us) client_negotiation.cc:769] Sending connection context
0619 08:23:57.070208 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 08:23:57.070332 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":129,"threads_started":1}
I20260619 08:23:57.071240 32141 heartbeater.cc:347] Connected to a master server at 127.25.127.190:41383
I20260619 08:23:57.071348 32141 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:57.071487 32141 heartbeater.cc:511] Master 127.25.127.190:41383 requested a full tablet report, sending...
I20260619 08:23:57.071885 31681 ts_manager.cc:194] Registered new tserver with Master: 2f0450a6d309480d94c8614481d97755 (127.25.127.131:36403)
I20260619 08:23:57.072307 31681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:47019
I20260619 08:23:57.082489 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/test-admin.keytab.
Jun 19 08:23:57 dist-test-slave-5rhg krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:57.096849 26110 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 08:23:57.098444 32148 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:23:57.097703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49156 (local address 127.25.127.129:38009)
0619 08:23:57.097867 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:57.097873 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:57.097890 (+ 17us) server_negotiation.cc:407] Connection header received
0619 08:23:57.097915 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:57.097916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:57.097932 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:57.098019 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:57.098360 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49156: BlockingRecv error: recv got EOF from 127.0.0.1:49156 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":67,"threads_started":1}
W20260619 08:23:57.098524 26110 init.cc:244] Error while opening credential cache '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/krb5cc)
Jun 19 08:23:57 dist-test-slave-5rhg krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:57.099511 26110 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 08:23:57 dist-test-slave-5rhg krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:57.100519 26110 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 08:23:57 dist-test-slave-5rhg krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.129@KRBTEST.COM
I20260619 08:23:57.108465 32148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:57.101037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49164 (local address 127.25.127.129:38009)
0619 08:23:57.101096 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:57.101098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:57.101132 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:23:57.101211 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:57.101211 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:57.101223 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:57.101288 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:57.101685 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:57.102407 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:57.103137 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:57.103477 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:57.103558 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:57.105643 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:57.105651 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:57.105651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:57.105657 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:57.107259 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:57.107600 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:57.107604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:57.107604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:57.107663 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:57.107851 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:57.107851 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:57.107852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:57.107979 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:57.108081 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:57.108235 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 08:23:57.108305 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260619 08:23:57.110222 32148 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:23:57.109630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49176 (local address 127.25.127.129:38009)
0619 08:23:57.109690 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:57.109693 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:57.109745 (+ 52us) server_negotiation.cc:407] Connection header received
0619 08:23:57.109828 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:57.109829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:57.109839 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:57.109894 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:57.110168 (+ 274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49176: BlockingRecv error: recv got EOF from 127.0.0.1:49176 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260619 08:23:57.110298 26110 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/krb5cc' of type FILE
Jun 19 08:23:57 dist-test-slave-5rhg krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:57.111212 26110 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 08:23:57 dist-test-slave-5rhg krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:57.112008 26110 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 08:23:57 dist-test-slave-5rhg krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.129@KRBTEST.COM
I20260619 08:23:57.118408 32148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:57.112541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49180 (local address 127.25.127.129:38009)
0619 08:23:57.112604 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:57.112606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:57.112614 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:23:57.112675 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:57.112676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:57.112696 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:57.112740 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:57.113078 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:57.113594 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:57.114234 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:57.114426 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:57.114475 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:57.116372 (+ 1897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:57.116373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:57.116373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:57.116379 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:57.117502 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:57.117794 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:57.117794 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:57.117794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:57.117841 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:57.117980 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:57.117981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:57.117981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:57.118058 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:57.118122 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:57.118216 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 08:23:57.118273 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260619 08:23:57.119789 32148 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:23:57.119344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49188 (local address 127.25.127.129:38009)
0619 08:23:57.119380 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:57.119382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:57.119414 (+ 32us) server_negotiation.cc:407] Connection header received
0619 08:23:57.119461 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:57.119461 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:57.119468 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:57.119509 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:57.119761 (+ 252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49188: BlockingRecv error: recv got EOF from 127.0.0.1:49188 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 08:23:57.119876 26110 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781857412505079-26110-0/krb5kdc/krb5cc' of type FILE
Jun 19 08:23:57 dist-test-slave-5rhg krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:57.120842 26110 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 08:23:57 dist-test-slave-5rhg krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:57.121640 26110 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 08:23:57 dist-test-slave-5rhg krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.129@KRBTEST.COM
I20260619 08:23:57.127974 32148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:57.122086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49204 (local address 127.25.127.129:38009)
0619 08:23:57.122162 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:57.122164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:57.122269 (+ 105us) server_negotiation.cc:407] Connection header received
0619 08:23:57.122296 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:57.122296 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:57.122304 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:57.122351 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:57.122673 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:57.123119 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:57.123755 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:57.123931 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:57.123979 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:57.125927 (+ 1948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:57.125928 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:57.125928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:57.125934 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:57.127046 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:57.127340 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:57.127340 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:57.127341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:57.127386 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:57.127539 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:57.127539 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:57.127540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:57.127625 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:57.127692 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:57.127789 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 08:23:57.127841 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 08:23:57.128741 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 31744
I20260619 08:23:57.133684 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 31878
I20260619 08:23:57.138329 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 32012
I20260619 08:23:57.142808 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 31650
2026-06-19T08:23:57Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1872 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:23:57 dist-test-slave-5rhg krb5kdc[32161](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:23:57 dist-test-slave-5rhg krb5kdc[32161](info): set up 2 sockets
Jun 19 08:23:57 dist-test-slave-5rhg krb5kdc[32161](info): commencing operation
krb5kdc: starting...
W20260619 08:23:58.185271 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:23:58 dist-test-slave-5rhg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:23:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:23:58Z Disabled control of system clock
WARNING: no policy specified for oryx/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.190.keytab.
Entry for principal oryx/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.190.keytab.
I20260619 08:23:58.304622 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:35483
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.190.keytab
--principal=oryx/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:37543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:35483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:58.387872 32174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:58.388170 32174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:58.388196 32174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:58.389321 32174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:23:58.389364 32174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:58.389379 32174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:58.389393 32174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:23:58.389407 32174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:23:58.390857 32174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:37543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:35483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.190.keytab
--principal=oryx/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:35483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:58.391080 32174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:58.391263 32174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:58.396332 32179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:58.396294 32182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:58.396293 32180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:58.396579 32174 server_base.cc:1034] running on GCE node
I20260619 08:23:58.396829 32174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:58.397156 32174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:58.398329 32174 hybrid_clock.cc:648] HybridClock initialized: now 1781857438398325 us; error 41 us; skew 500 ppm
Jun 19 08:23:58 dist-test-slave-5rhg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857438, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:58.401072 32174 init.cc:377] Logged in from keytab as oryx/127.25.127.190@KRBTEST.COM (short username oryx)
I20260619 08:23:58.401583 32174 webserver.cc:460] Webserver started at http://127.25.127.190:37241/ using document root <none> and password file <none>
I20260619 08:23:58.401814 32174 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:58.401878 32174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:58.402024 32174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:58.403131 32174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "a382c3e1cdce42d89abf62d01255925d"
format_stamp: "Formatted at 2026-06-19 08:23:58 on dist-test-slave-5rhg"
server_key: "7ed77e1c811f15bf01cc0f646cc32c1c"
server_key_iv: "0848a00ef0d7f9ea578df485e584f405"
server_key_version: "encryptionkey@0"
I20260619 08:23:58.403515 32174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "a382c3e1cdce42d89abf62d01255925d"
format_stamp: "Formatted at 2026-06-19 08:23:58 on dist-test-slave-5rhg"
server_key: "7ed77e1c811f15bf01cc0f646cc32c1c"
server_key_iv: "0848a00ef0d7f9ea578df485e584f405"
server_key_version: "encryptionkey@0"
I20260619 08:23:58.404985 32174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:23:58.405767 32189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:58.405954 32174 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:58.406059 32174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "a382c3e1cdce42d89abf62d01255925d"
format_stamp: "Formatted at 2026-06-19 08:23:58 on dist-test-slave-5rhg"
server_key: "7ed77e1c811f15bf01cc0f646cc32c1c"
server_key_iv: "0848a00ef0d7f9ea578df485e584f405"
server_key_version: "encryptionkey@0"
I20260619 08:23:58.406134 32174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:58.425685 32174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:58.427815 32174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:58.427992 32174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:58.432220 32174 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:35483
I20260619 08:23:58.432276 32241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:35483 every 8 connection(s)
I20260619 08:23:58.432660 32174 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:23:58.433698 32242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:58.435976 32242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d: Bootstrap starting.
I20260619 08:23:58.436564 32242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:58.436805 32242 log.cc:824] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:58.437506 32242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d: No bootstrap required, opened a new log
I20260619 08:23:58.439066 32242 raft_consensus.cc:348] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } }
I20260619 08:23:58.439208 32242 raft_consensus.cc:374] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:58.439230 32242 raft_consensus.cc:729] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a382c3e1cdce42d89abf62d01255925d, State: Initialized, Role: FOLLOWER
I20260619 08:23:58.439332 32242 consensus_queue.cc:260] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } }
I20260619 08:23:58.439417 32242 raft_consensus.cc:388] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:23:58.439464 32242 raft_consensus.cc:482] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:23:58.439548 32242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:58.439815 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 32174
I20260619 08:23:58.439917 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:23:58.440105 26110 external_mini_cluster.cc:1428] Setting key 54fd5436ab353f952be6254e46e90636
I20260619 08:23:58.440809 32242 raft_consensus.cc:504] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } }
I20260619 08:23:58.440930 32242 leader_election.cc:302] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a382c3e1cdce42d89abf62d01255925d; no voters:
I20260619 08:23:58.441102 32242 leader_election.cc:288] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:23:58.441154 32247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:23:58.441327 32247 raft_consensus.cc:686] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [term 1 LEADER]: Becoming Leader. State: Replica: a382c3e1cdce42d89abf62d01255925d, State: Running, Role: LEADER
I20260619 08:23:58.441386 32242 sys_catalog.cc:564] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:58.441431 32247 consensus_queue.cc:237] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } }
I20260619 08:23:58.441790 32249 sys_catalog.cc:455] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } }
I20260619 08:23:58.441895 32249 sys_catalog.cc:458] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:58.441783 32247 sys_catalog.cc:455] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a382c3e1cdce42d89abf62d01255925d. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } }
I20260619 08:23:58.442147 32247 sys_catalog.cc:458] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:58.442271 32255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:23:58.443041 32255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:23:58.444952 32255 catalog_manager.cc:1347] Generated new cluster ID: 5f7f4708a0c94aab8e7bf11e4f6f51f7
I20260619 08:23:58.445007 32255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:23:58 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM
I20260619 08:23:58.453081 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:58.440718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43374 (local address 127.25.127.190:35483)
0619 08:23:58.441013 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:58.441023 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:58.441064 (+ 41us) server_negotiation.cc:407] Connection header received
0619 08:23:58.441215 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:58.441224 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:58.441291 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:58.442283 (+ 992us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:58.442395 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:58.443848 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:58.444931 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:58.445577 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:58.445706 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:58.448565 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:58.448569 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:58.448570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:58.448583 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:58.451240 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:58.451672 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:58.451675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:58.451675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:58.451810 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:58.452035 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:58.452035 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:58.452036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:58.452398 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:58.452561 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:58.452690 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 08:23:58.452831 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":96,"threads_started":1}
I20260619 08:23:58.460743 32255 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:23:58.461467 32255 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:23:58.466387 32255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d: Generated new TSK 0
I20260619 08:23:58.466584 32255 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.129.keytab.
Entry for principal oryx/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.129.keytab.
I20260619 08:23:58.493209 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:35483
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.129.keytab
--principal=oryx/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:37543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:58.580385 32268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:58.580614 32268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:58.580641 32268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:58.581873 32268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:58.581918 32268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:58.581947 32268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:23:58.583465 32268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:37543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.129.keytab
--principal=oryx/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:35483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:58.583758 32268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:58.584023 32268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:23:58.589303 32268 server_base.cc:1034] running on GCE node
W20260619 08:23:58.589298 32273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:58.589277 32274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:58.589283 32276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:58.589706 32268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:58.589978 32268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:58.591125 32268 hybrid_clock.cc:648] HybridClock initialized: now 1781857438591099 us; error 48 us; skew 500 ppm
Jun 19 08:23:58 dist-test-slave-5rhg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857438, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:58.593818 32268 init.cc:377] Logged in from keytab as oryx/127.25.127.129@KRBTEST.COM (short username oryx)
I20260619 08:23:58.594298 32268 webserver.cc:460] Webserver started at http://127.25.127.129:33625/ using document root <none> and password file <none>
I20260619 08:23:58.594534 32268 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:58.594579 32268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:58.594745 32268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:58.595809 32268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "e891c0b19a604149b7038f924fb83195"
format_stamp: "Formatted at 2026-06-19 08:23:58 on dist-test-slave-5rhg"
server_key: "2dd25c02ae98de3258a2b74a3da3d9a7"
server_key_iv: "05fecc61260170525fa9443bf993df11"
server_key_version: "encryptionkey@0"
I20260619 08:23:58.596230 32268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "e891c0b19a604149b7038f924fb83195"
format_stamp: "Formatted at 2026-06-19 08:23:58 on dist-test-slave-5rhg"
server_key: "2dd25c02ae98de3258a2b74a3da3d9a7"
server_key_iv: "05fecc61260170525fa9443bf993df11"
server_key_version: "encryptionkey@0"
I20260619 08:23:58.597533 32268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:58.598315 32283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:58.598475 32268 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:58.598579 32268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "e891c0b19a604149b7038f924fb83195"
format_stamp: "Formatted at 2026-06-19 08:23:58 on dist-test-slave-5rhg"
server_key: "2dd25c02ae98de3258a2b74a3da3d9a7"
server_key_iv: "05fecc61260170525fa9443bf993df11"
server_key_version: "encryptionkey@0"
I20260619 08:23:58.598673 32268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:58.614377 32268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:58.616618 32268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:58.616804 32268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:58.617057 32268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:58.617399 32268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:58.617460 32268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:58.617502 32268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:58.617532 32268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:58.623904 32268 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:41471
I20260619 08:23:58.623951 32396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:41471 every 8 connection(s)
I20260619 08:23:58.624315 32268 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:23:58.628517 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 32268
I20260619 08:23:58.628640 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:23:58.628844 26110 external_mini_cluster.cc:1428] Setting key 07f8762884b2f41872889d601789f38d
Jun 19 08:23:58 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857438, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM
I20260619 08:23:58.635144 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:58.625093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:44897 (local address 127.25.127.190:35483)
0619 08:23:58.625167 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:58.625169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:58.625392 (+ 223us) server_negotiation.cc:407] Connection header received
0619 08:23:58.625965 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:58.625966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:58.625985 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:58.626038 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:58.627039 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:58.627567 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:58.628760 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:58.629030 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:58.629103 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:58.631955 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:58.631956 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:58.631957 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:58.631965 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:58.633514 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:58.634021 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:58.634022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:58.634022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:58.634072 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:58.634383 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:58.634384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:58.634384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:58.634492 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:58.634538 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:58.634947 (+ 409us) server_negotiation.cc:299] Negotiation successful
0619 08:23:58.635043 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 08:23:58.635371 32399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:58.625076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.129:44897)
0619 08:23:58.625265 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:58.625304 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:58.625843 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:58.626115 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:58.626119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:58.626724 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:58.626931 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:58.626937 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:58.627672 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:58.627673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:58.628621 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:58.628624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:58.629193 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:58.629194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:58.629347 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:58.629931 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:58.629943 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:58.631849 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:58.633597 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:58.633606 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:58.633609 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:58.633941 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:58.634149 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:58.634150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:58.634151 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:58.634309 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:58.634556 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:58.634563 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:58.634763 (+ 200us) client_negotiation.cc:769] Sending connection context
0619 08:23:58.634898 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 08:23:58.635096 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260619 08:23:58.635923 32397 heartbeater.cc:347] Connected to a master server at 127.25.127.190:35483
I20260619 08:23:58.636053 32397 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:58.636286 32397 heartbeater.cc:511] Master 127.25.127.190:35483 requested a full tablet report, sending...
I20260619 08:23:58.636874 32206 ts_manager.cc:194] Registered new tserver with Master: e891c0b19a604149b7038f924fb83195 (127.25.127.129:41471)
WARNING: no policy specified for oryx/127.25.127.130@KRBTEST.COM; defaulting to no policy
I20260619 08:23:58.637513 32206 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:44897
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.127.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.130.keytab.
Entry for principal oryx/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.130.keytab.
I20260619 08:23:58.655951 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:35483
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.130.keytab
--principal=oryx/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:37543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:58.739473 32402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:58.739688 32402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:58.739712 32402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:58.740988 32402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:58.741042 32402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:58.741070 32402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:23:58.742856 32402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:37543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.130.keytab
--principal=oryx/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:35483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:58.743121 32402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:58.743362 32402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:58.748428 32410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:58.748576 32402 server_base.cc:1034] running on GCE node
W20260619 08:23:58.748428 32408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:58.748483 32407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:58.748862 32402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:58.749079 32402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:58.750226 32402 hybrid_clock.cc:648] HybridClock initialized: now 1781857438750211 us; error 25 us; skew 500 ppm
Jun 19 08:23:58 dist-test-slave-5rhg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857438, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:58.752976 32402 init.cc:377] Logged in from keytab as oryx/127.25.127.130@KRBTEST.COM (short username oryx)
I20260619 08:23:58.753491 32402 webserver.cc:460] Webserver started at http://127.25.127.130:36279/ using document root <none> and password file <none>
I20260619 08:23:58.753727 32402 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:58.753795 32402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:58.753959 32402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:58.755043 32402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "15f897dd709d42f48d6b51b65720e9bc"
format_stamp: "Formatted at 2026-06-19 08:23:58 on dist-test-slave-5rhg"
server_key: "69697cf450e06729cc17f3007534cfc0"
server_key_iv: "29a8531055af6d6af580c8c2f34fbe94"
server_key_version: "encryptionkey@0"
I20260619 08:23:58.755426 32402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "15f897dd709d42f48d6b51b65720e9bc"
format_stamp: "Formatted at 2026-06-19 08:23:58 on dist-test-slave-5rhg"
server_key: "69697cf450e06729cc17f3007534cfc0"
server_key_iv: "29a8531055af6d6af580c8c2f34fbe94"
server_key_version: "encryptionkey@0"
I20260619 08:23:58.756845 32402 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:23:58.757709 32417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:58.757906 32402 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:23:58.758025 32402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "15f897dd709d42f48d6b51b65720e9bc"
format_stamp: "Formatted at 2026-06-19 08:23:58 on dist-test-slave-5rhg"
server_key: "69697cf450e06729cc17f3007534cfc0"
server_key_iv: "29a8531055af6d6af580c8c2f34fbe94"
server_key_version: "encryptionkey@0"
I20260619 08:23:58.758137 32402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:58.805181 32402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:58.807802 32402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:58.807986 32402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:58.808279 32402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:58.808645 32402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:58.808701 32402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:58.808745 32402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:58.808792 32402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:58.814749 32402 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:34349
I20260619 08:23:58.814796 32530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:34349 every 8 connection(s)
I20260619 08:23:58.815143 32402 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:23:58.821683 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 32402
Jun 19 08:23:58 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857438, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM
I20260619 08:23:58.821811 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:23:58.822022 26110 external_mini_cluster.cc:1428] Setting key 434356de7aca4d03e63dd92a5f1ee5ea
I20260619 08:23:58.825896 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:58.815956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:50251 (local address 127.25.127.190:35483)
0619 08:23:58.816138 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:58.816140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:58.816395 (+ 255us) server_negotiation.cc:407] Connection header received
0619 08:23:58.816986 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:58.816987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:58.817000 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:58.817049 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:58.818051 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:58.818587 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:58.819611 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:58.819814 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:58.819904 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:58.822518 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:58.822519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:58.822520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:58.822529 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:58.824356 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:58.824800 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:58.824801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:58.824802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:58.824866 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:58.825192 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:58.825194 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:58.825194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:58.825323 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:58.825407 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:58.825710 (+ 303us) server_negotiation.cc:299] Negotiation successful
0619 08:23:58.825804 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260619 08:23:58.826095 32533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:58.815930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.130:50251)
0619 08:23:58.816142 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:58.816289 (+ 147us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:58.816866 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:58.817128 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:58.817132 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:58.817720 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:58.817927 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:58.817932 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:58.818689 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:58.818691 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:58.819513 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:58.819515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:58.819913 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:58.819914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:58.820052 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:58.820574 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:58.820583 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:58.822393 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:58.824462 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:58.824469 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:58.824471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:58.824715 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:58.824966 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:58.824968 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:58.824968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:58.825102 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:58.825397 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:58.825404 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:58.825561 (+ 157us) client_negotiation.cc:769] Sending connection context
0619 08:23:58.825674 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 08:23:58.825846 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":88,"threads_started":1}
I20260619 08:23:58.826772 32531 heartbeater.cc:347] Connected to a master server at 127.25.127.190:35483
I20260619 08:23:58.826902 32531 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:58.827122 32531 heartbeater.cc:511] Master 127.25.127.190:35483 requested a full tablet report, sending...
I20260619 08:23:58.827564 32206 ts_manager.cc:194] Registered new tserver with Master: 15f897dd709d42f48d6b51b65720e9bc (127.25.127.130:34349)
I20260619 08:23:58.828104 32206 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:50251
WARNING: no policy specified for oryx/127.25.127.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.131.keytab.
Entry for principal oryx/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.131.keytab.
I20260619 08:23:58.848661 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:35483
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.131.keytab
--principal=oryx/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:37543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:58.931236 32536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:58.931488 32536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:58.931529 32536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:58.932767 32536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:58.932843 32536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:58.932893 32536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:23:58.934450 32536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:37543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.131.keytab
--principal=oryx/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:35483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:58.934738 32536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:58.934983 32536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:58.940210 32541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:58.940269 32536 server_base.cc:1034] running on GCE node
W20260619 08:23:58.940254 32544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:58.940325 32542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:58.940770 32536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:58.941093 32536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:58.942246 32536 hybrid_clock.cc:648] HybridClock initialized: now 1781857438942237 us; error 34 us; skew 500 ppm
Jun 19 08:23:58 dist-test-slave-5rhg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857438, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:58.945127 32536 init.cc:377] Logged in from keytab as oryx/127.25.127.131@KRBTEST.COM (short username oryx)
I20260619 08:23:58.945576 32536 webserver.cc:460] Webserver started at http://127.25.127.131:37765/ using document root <none> and password file <none>
I20260619 08:23:58.945837 32536 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:58.945883 32536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:58.946025 32536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:58.947121 32536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "2d9893d7478843c3b96c5b842fc4d11f"
format_stamp: "Formatted at 2026-06-19 08:23:58 on dist-test-slave-5rhg"
server_key: "b5ee6e3f85023a9c458c51062f915c5d"
server_key_iv: "80f384a24daf1053ae1cc089994fdc89"
server_key_version: "encryptionkey@0"
I20260619 08:23:58.947477 32536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "2d9893d7478843c3b96c5b842fc4d11f"
format_stamp: "Formatted at 2026-06-19 08:23:58 on dist-test-slave-5rhg"
server_key: "b5ee6e3f85023a9c458c51062f915c5d"
server_key_iv: "80f384a24daf1053ae1cc089994fdc89"
server_key_version: "encryptionkey@0"
I20260619 08:23:58.948850 32536 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:23:58.949599 32551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:58.949777 32536 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:58.949857 32536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "2d9893d7478843c3b96c5b842fc4d11f"
format_stamp: "Formatted at 2026-06-19 08:23:58 on dist-test-slave-5rhg"
server_key: "b5ee6e3f85023a9c458c51062f915c5d"
server_key_iv: "80f384a24daf1053ae1cc089994fdc89"
server_key_version: "encryptionkey@0"
I20260619 08:23:58.949913 32536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:58.972078 32536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:58.974614 32536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:58.974728 32536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:58.974920 32536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:58.975189 32536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:58.975222 32536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:58.975245 32536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:58.975258 32536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:58.981140 32536 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:39721
I20260619 08:23:58.981185 32664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:39721 every 8 connection(s)
I20260619 08:23:58.981525 32536 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:23:58.983790 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 32536
I20260619 08:23:58.983901 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:23:58.984073 26110 external_mini_cluster.cc:1428] Setting key 9fc44415af2810b66fa67b2c05bb7677
Jun 19 08:23:58 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857438, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM
I20260619 08:23:58.990968 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:58.982286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:42569 (local address 127.25.127.190:35483)
0619 08:23:58.982351 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:58.982353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:58.982583 (+ 230us) server_negotiation.cc:407] Connection header received
0619 08:23:58.983052 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:58.983053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:58.983063 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:58.983131 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:58.984143 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:58.984916 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:58.985963 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:58.986152 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:58.986208 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:58.988313 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:58.988315 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:58.988315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:58.988322 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:58.989812 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:58.990167 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:58.990168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:58.990168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:58.990219 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:58.990428 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:58.990428 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:58.990428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:58.990529 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:58.990580 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:58.990839 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 08:23:58.990900 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:23:58.991146 32667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:58.982267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.131:42569)
0619 08:23:58.982442 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:58.982486 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:58.982973 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:58.983189 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:58.983192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:58.983772 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:58.984042 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:58.984048 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:58.984999 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:58.985000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:58.985887 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:58.985890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:58.986229 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:58.986229 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:58.986332 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:58.986801 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:58.986810 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:58.988229 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:58.989883 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:58.989889 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:58.989890 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:58.990100 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:58.990276 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:58.990276 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:58.990276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:58.990375 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:58.990581 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:58.990585 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:58.990727 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 08:23:58.990812 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 08:23:58.990936 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260619 08:23:58.991636 32665 heartbeater.cc:347] Connected to a master server at 127.25.127.190:35483
I20260619 08:23:58.991726 32665 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:58.991878 32665 heartbeater.cc:511] Master 127.25.127.190:35483 requested a full tablet report, sending...
I20260619 08:23:58.992281 32206 ts_manager.cc:194] Registered new tserver with Master: 2d9893d7478843c3b96c5b842fc4d11f (127.25.127.131:39721)
I20260619 08:23:58.992621 32206 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:42569
I20260619 08:23:58.995826 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.25.127.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.127.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.127.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.189.keytab.
Entry for principal oryx/127.25.127.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.189.keytab.
I20260619 08:23:59.020750 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.189:46777
--webserver_interface=127.25.127.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.189.keytab
--principal=oryx/127.25.127.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:37543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:35483,127.25.127.189:46777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:59.102717 32670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:59.102927 32670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:59.102950 32670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:59.104054 32670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:23:59.104089 32670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:59.104103 32670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:59.104116 32670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:23:59.104128 32670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:23:59.105625 32670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:37543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:35483,127.25.127.189:46777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.189.keytab
--principal=oryx/127.25.127.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.189:46777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.127.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
W20260619 08:23:59.105727 32670 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.25.127.190:35483,127.25.127.189:46777'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 08:23:59.105837 32670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:59.106000 32670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:59.110953 32678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:59.111017 32670 server_base.cc:1034] running on GCE node
W20260619 08:23:59.110948 32675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:59.110958 32676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:59.111370 32670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:59.111685 32670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:59.112865 32670 hybrid_clock.cc:648] HybridClock initialized: now 1781857439112848 us; error 27 us; skew 500 ppm
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:59.115489 32670 init.cc:377] Logged in from keytab as oryx/127.25.127.189@KRBTEST.COM (short username oryx)
I20260619 08:23:59.115928 32670 webserver.cc:460] Webserver started at http://127.25.127.189:38593/ using document root <none> and password file <none>
I20260619 08:23:59.116250 32670 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:59.116320 32670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:59.116479 32670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:59.117583 32670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/data/instance:
uuid: "473f4c6d5426459a9c73ab26656abe8d"
format_stamp: "Formatted at 2026-06-19 08:23:59 on dist-test-slave-5rhg"
server_key: "e272cfc7862537692792f8d406d9375d"
server_key_iv: "bc3f57e8731d61b6a225a5b4c43df864"
server_key_version: "encryptionkey@0"
I20260619 08:23:59.117956 32670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/wal/instance:
uuid: "473f4c6d5426459a9c73ab26656abe8d"
format_stamp: "Formatted at 2026-06-19 08:23:59 on dist-test-slave-5rhg"
server_key: "e272cfc7862537692792f8d406d9375d"
server_key_iv: "bc3f57e8731d61b6a225a5b4c43df864"
server_key_version: "encryptionkey@0"
I20260619 08:23:59.119282 32670 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:23:59.119957 32685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.120126 32670 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:59.120268 32670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/wal
uuid: "473f4c6d5426459a9c73ab26656abe8d"
format_stamp: "Formatted at 2026-06-19 08:23:59 on dist-test-slave-5rhg"
server_key: "e272cfc7862537692792f8d406d9375d"
server_key_iv: "bc3f57e8731d61b6a225a5b4c43df864"
server_key_version: "encryptionkey@0"
I20260619 08:23:59.120352 32670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:59.128376 32670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:59.130681 32670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:59.130860 32670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:59.134654 32670 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.189:46777
I20260619 08:23:59.134714 32737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.189:46777 every 8 connection(s)
I20260619 08:23:59.135085 32670 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/data/info.pb
I20260619 08:23:59.135404 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 32670
I20260619 08:23:59.136410 32738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:59.138325 32738 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.127.189@KRBTEST.COM
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.189@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM
I20260619 08:23:59.147187 32741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.136051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41166 (local address 127.25.127.189:46777)
0619 08:23:59.136260 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.136268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.136292 (+ 24us) server_negotiation.cc:407] Connection header received
0619 08:23:59.136797 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.136804 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.136853 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.136978 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.137751 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.138790 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.139869 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.140598 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.140728 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.143502 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.143509 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.143509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.143515 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.145543 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.145936 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.145940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.145941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.146041 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.146251 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.146252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.146252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.146566 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.146720 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.146817 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.146954 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":109,"threads_started":1}
I20260619 08:23:59.147244 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.138984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43386 (local address 127.25.127.190:35483)
0619 08:23:59.139114 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.139116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.139215 (+ 99us) server_negotiation.cc:407] Connection header received
0619 08:23:59.139279 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.139280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.139289 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.139331 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.139872 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.140599 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.141768 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.141954 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.142013 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.144468 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.144469 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.144470 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.144478 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.145928 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.146368 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.146369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.146370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.146424 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.146589 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.146590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.146590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.146780 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.146847 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.147048 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.147143 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260619 08:23:59.147290 32742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.138919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:43386)
0619 08:23:59.139079 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.139145 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.139218 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.139357 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.139360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.139682 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.139765 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.139767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.140704 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.140705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.141680 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.141683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.142038 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.142039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.142135 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.142714 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.142721 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.144358 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.146006 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.146012 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.146014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.146297 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.146487 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.146488 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.146488 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.146534 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.146866 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.146874 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.146927 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.147017 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.147143 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260619 08:23:59.148046 32738 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.189@KRBTEST.COM for oryx/127.25.127.189@KRBTEST.COM
I20260619 08:23:59.154356 32742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.148328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:46777 (local address 127.0.0.1:41174)
0619 08:23:59.148377 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.148389 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.148435 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.148553 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.148554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.148686 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.148728 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.148730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.149370 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.149370 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.149847 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.149848 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.150233 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.150234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.150295 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.150665 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.150669 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.151827 (+ 1158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.153297 (+ 1470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.153298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.153298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.153454 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.153678 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.153679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.153680 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.153732 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.153997 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.153998 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.154035 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.154147 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.154246 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 08:23:59.154399 32741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.148379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41174 (local address 127.25.127.189:46777)
0619 08:23:59.148407 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.148408 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.148419 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:23:59.148489 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.148489 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.148498 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.148551 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.148806 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.149280 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.149938 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.150151 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.150233 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.151920 (+ 1687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.151921 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.151922 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.151928 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.153209 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.153552 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.153553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.153553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.153603 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.153815 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.153815 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.153816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.153916 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.153998 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.154171 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.154294 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":14}
I20260619 08:23:59.155735 32738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: Bootstrap starting.
I20260619 08:23:59.156591 32738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:59.156821 32738 log.cc:824] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:59.157474 32738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: No bootstrap required, opened a new log
I20260619 08:23:59.158800 32738 raft_consensus.cc:348] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } }
I20260619 08:23:59.158918 32738 raft_consensus.cc:374] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:59.158941 32738 raft_consensus.cc:729] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 473f4c6d5426459a9c73ab26656abe8d, State: Initialized, Role: FOLLOWER
I20260619 08:23:59.159029 32738 consensus_queue.cc:260] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } }
I20260619 08:23:59.159338 32738 sys_catalog.cc:564] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:59.159269 32744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } } }
I20260619 08:23:59.159402 32744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [sys.catalog]: This master's current role is: FOLLOWER
W20260619 08:23:59.161268 32755 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:23:59.161332 32755 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:23:59.162988 32670 master_runner.cc:428] Detected that this master 473f4c6d5426459a9c73ab26656abe8d is joining an existing cluster
I20260619 08:23:59.163064 32670 master_runner.cc:432] Initiating AddMaster RPC to add 127.25.127.189:46777
I20260619 08:23:59.171687 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.164343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43402 (local address 127.25.127.190:35483)
0619 08:23:59.164399 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.164400 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.164424 (+ 24us) server_negotiation.cc:407] Connection header received
0619 08:23:59.164530 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.164531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.164545 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.164576 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.165212 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.165710 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.166830 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.167057 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.167089 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.168599 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.168601 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.168601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.168608 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.170296 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.170809 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.170810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.170811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.170872 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.171101 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.171102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.171102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.171234 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.171269 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.171509 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.171598 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 08:23:59.171780 32741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.164330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41180 (local address 127.25.127.189:46777)
0619 08:23:59.164379 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.164381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.164552 (+ 171us) server_negotiation.cc:407] Connection header received
0619 08:23:59.164595 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.164596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.164606 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.164681 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.165189 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.165920 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.166856 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.167150 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.167222 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.168575 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.168576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.168577 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.168584 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.170460 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.170864 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.170865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.170866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.170935 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.171139 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.171140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.171140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.171268 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.171339 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.171485 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.171664 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":88,"server-negotiator.queue_time_us":25}
I20260619 08:23:59.172034 32762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.164126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:43402)
0619 08:23:59.164344 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.164375 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.164421 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.164680 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.164680 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.165011 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.165107 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.165109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.165806 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.165806 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.166725 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.166727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.167164 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.167164 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.167243 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.167897 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.167901 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.168497 (+ 596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.170402 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.170403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.170403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.170717 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.170945 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.170946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.170946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.171023 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.171327 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.171327 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.171377 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.171834 (+ 457us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.171949 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":8,"thread_start_us":127,"threads_started":1}
I20260619 08:23:59.171700 32761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.164127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:46777 (local address 127.0.0.1:41180)
0619 08:23:59.164469 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.164479 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.164543 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.164692 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.164693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.164996 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.165108 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.165109 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.166005 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.166006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.166775 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.166776 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.167231 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.167232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.167299 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.168033 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.168037 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.168484 (+ 447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.170538 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.170538 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.170539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.170790 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.171019 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.171019 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.171020 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.171075 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.171341 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.171342 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.171377 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.171451 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.171578 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":2,"thread_start_us":141,"threads_started":1}
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857438, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.190@KRBTEST.COM for oryx/127.25.127.189@KRBTEST.COM
I20260619 08:23:59.182128 32763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.174463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:46777 (local address 127.0.0.1:41196)
0619 08:23:59.174663 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.174694 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.174752 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.174889 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.174891 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.175151 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.175200 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.175203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.176115 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.176116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.176798 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.176801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.177261 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.177262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.177339 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.177785 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.177789 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.179168 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.181007 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.181015 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.181016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.181222 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.181450 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.181450 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.181451 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.181515 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.181808 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.181814 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.181865 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.181939 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.182042 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":86,"threads_started":1}
I20260619 08:23:59.182154 32741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.174537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41196 (local address 127.25.127.189:46777)
0619 08:23:59.174575 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.174577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.174753 (+ 176us) server_negotiation.cc:407] Connection header received
0619 08:23:59.174808 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.174809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.174819 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.174879 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.175270 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.175998 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.176878 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.177178 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.177249 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.179278 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.179279 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.179279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.179286 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.180905 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.181296 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.181297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.181298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.181357 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.181591 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.181592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.181592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.181735 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.181802 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.181982 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.182060 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 08:23:59.182991 32206 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.25.127.189:46777: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d"
member_type: NON_VOTER
last_known_addr {
host: "127.25.127.189"
port: 46777
}
attrs {
promote: true
}
}
dest_uuid: "a382c3e1cdce42d89abf62d01255925d"
cas_config_opid_index: -1
I20260619 08:23:59.183324 32206 consensus_queue.cc:237] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } }
I20260619 08:23:59.183869 32765 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 473f4c6d5426459a9c73ab26656abe8d (127.25.127.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } } }
I20260619 08:23:59.184190 32712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:59.185003 32765 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.25.127.189:46777
I20260619 08:23:59.185055 32712 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 FOLLOWER]: Refusing update from remote peer a382c3e1cdce42d89abf62d01255925d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 08:23:59.185155 32765 sys_catalog.cc:455] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } } }
I20260619 08:23:59.185221 32765 sys_catalog.cc:458] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:59.185307 32670 master.cc:537] Master@127.25.127.189:46777 shutting down...
I20260619 08:23:59.185840 32765 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [LEADER]: Connected to new peer: Peer: permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 08:23:59.186523 32191 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260619 08:23:59.186993 32191 consensus_peers.cc:487] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d -> Peer 473f4c6d5426459a9c73ab26656abe8d (127.25.127.189:46777): Couldn't send request to peer 473f4c6d5426459a9c73ab26656abe8d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 08:23:59.188470 32670 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:23:59.188571 32670 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:23:59.188622 32670 tablet_replica.cc:330] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: stopping tablet replica
I20260619 08:23:59.201879 32670 master.cc:552] Master@127.25.127.189:46777 shutdown complete.
I20260619 08:23:59.201960 32670 master_runner.cc:305] Clearing existing system tablet
I20260619 08:23:59.202109 32670 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:59.202178 32670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:59.202967 32670 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.000s
I20260619 08:23:59.203544 301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.203681 32670 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:59.203745 32670 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.203773 32670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/wal
uuid: "473f4c6d5426459a9c73ab26656abe8d"
format_stamp: "Formatted at 2026-06-19 08:23:59 on dist-test-slave-5rhg"
server_key: "e272cfc7862537692792f8d406d9375d"
server_key_iv: "bc3f57e8731d61b6a225a5b4c43df864"
server_key_version: "encryptionkey@0"
I20260619 08:23:59.204574 32670 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:59.206032 32670 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 08:23:59.206137 32670 log.cc:1196] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 08:23:59.206369 32670 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: Deleting consensus metadata
I20260619 08:23:59.206511 32670 master_runner.cc:315] Copying system tablet from 127.25.127.190:35483
I20260619 08:23:59.207183 32670 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: tablet copy: Beginning tablet copy session from remote peer at address 127.25.127.190:35483
I20260619 08:23:59.214517 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.207603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43412 (local address 127.25.127.190:35483)
0619 08:23:59.207658 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.207661 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.207799 (+ 138us) server_negotiation.cc:407] Connection header received
0619 08:23:59.207876 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.207877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.207889 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.207960 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.208384 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.208940 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.210127 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.210389 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.210457 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.211586 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.211588 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.211588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.211602 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.213356 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.213822 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.213822 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.213823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.213880 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.214071 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.214072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.214072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.214179 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.214228 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.214392 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.214451 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 08:23:59.214594 307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.207541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:43412)
0619 08:23:59.207718 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.207728 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.207817 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.207963 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.207964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.208211 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.208286 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.208289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.209063 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.209064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.209983 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.209986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.210467 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.210468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.210545 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.211058 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.211064 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.211469 (+ 405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.213455 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.213456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.213457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.213744 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.213943 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.213944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.213944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.214011 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.214239 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.214240 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.214289 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.214378 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.214485 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":81,"threads_started":1}
I20260619 08:23:59.214936 32226 tablet_copy_service.cc:140] P a382c3e1cdce42d89abf62d01255925d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 473f4c6d5426459a9c73ab26656abe8d ({username='oryx', principal='oryx/127.25.127.189@KRBTEST.COM'} at 127.0.0.1:43412)
I20260619 08:23:59.215049 32226 tablet_copy_service.cc:161] P a382c3e1cdce42d89abf62d01255925d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 473f4c6d5426459a9c73ab26656abe8d at {username='oryx', principal='oryx/127.25.127.189@KRBTEST.COM'} at 127.0.0.1:43412: session id = 473f4c6d5426459a9c73ab26656abe8d-00000000000000000000000000000000
I20260619 08:23:59.215891 32226 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 08:23:59.216595 32670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 08:23:59.217588 32670 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: tablet copy: Starting download of 0 data blocks...
I20260619 08:23:59.217758 32670 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: tablet copy: Starting download of 1 WAL segments...
I20260619 08:23:59.218619 32670 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 08:23:59.219436 32226 tablet_copy_service.cc:342] P a382c3e1cdce42d89abf62d01255925d: Request end of tablet copy session 473f4c6d5426459a9c73ab26656abe8d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.25.127.189@KRBTEST.COM'} at 127.0.0.1:43412
I20260619 08:23:59.219496 32226 tablet_copy_service.cc:433] P a382c3e1cdce42d89abf62d01255925d: ending tablet copy session 473f4c6d5426459a9c73ab26656abe8d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 473f4c6d5426459a9c73ab26656abe8d
I20260619 08:23:59.222050 32670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:59.223307 308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:59.223313 309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:59.223503 311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:59.224290 32670 server_base.cc:1034] running on GCE node
I20260619 08:23:59.224397 32670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:59.224532 32670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:59.225661 32670 hybrid_clock.cc:648] HybridClock initialized: now 1781857439225659 us; error 45 us; skew 500 ppm
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:59.226963 32670 init.cc:377] Logged in from keytab as oryx/127.25.127.189@KRBTEST.COM (short username oryx)
I20260619 08:23:59.227258 32670 webserver.cc:460] Webserver started at http://127.25.127.189:44187/ using document root <none> and password file <none>
I20260619 08:23:59.227384 32670 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:59.227432 32670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:59.228050 32670 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:59.228626 318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.228824 32670 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.228896 32670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/wal
uuid: "473f4c6d5426459a9c73ab26656abe8d"
format_stamp: "Formatted at 2026-06-19 08:23:59 on dist-test-slave-5rhg"
server_key: "e272cfc7862537692792f8d406d9375d"
server_key_iv: "bc3f57e8731d61b6a225a5b4c43df864"
server_key_version: "encryptionkey@0"
I20260619 08:23:59.228960 32670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:59.241258 32670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:59.243162 32670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:59.246341 32670 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.189:46777
I20260619 08:23:59.246752 32670 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-1/data/info.pb
I20260619 08:23:59.246757 370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.189:46777 every 8 connection(s)
I20260619 08:23:59.247707 371 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:23:59.248371 371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: Bootstrap starting.
I20260619 08:23:59.251009 371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:23:59.251258 371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: Bootstrap complete.
I20260619 08:23:59.251456 371 raft_consensus.cc:348] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } }
I20260619 08:23:59.251542 371 raft_consensus.cc:729] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 473f4c6d5426459a9c73ab26656abe8d, State: Initialized, Role: LEARNER
I20260619 08:23:59.251587 371 consensus_queue.cc:260] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } }
I20260619 08:23:59.251755 371 sys_catalog.cc:564] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:59.251864 375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } } }
I20260619 08:23:59.252030 375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [sys.catalog]: This master's current role is: LEARNER
I20260619 08:23:59.253731 386 catalog_manager.cc:1259] Loaded cluster ID: 5f7f4708a0c94aab8e7bf11e4f6f51f7
I20260619 08:23:59.253782 386 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: loading cluster ID for follower catalog manager: success
I20260619 08:23:59.254879 386 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: acquiring CA information for follower catalog manager: success
I20260619 08:23:59.255199 386 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 08:23:59.257261 373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.248912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41214 (local address 127.25.127.189:46777)
0619 08:23:59.249049 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.249052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.249113 (+ 61us) server_negotiation.cc:407] Connection header received
0619 08:23:59.249196 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.249197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.249208 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.249271 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.249861 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.250902 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.251982 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.252405 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.252477 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.254036 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.254037 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.254038 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.254045 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.255997 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.256468 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.256470 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.256470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.256542 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.256736 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.256737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.256737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.256874 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.256979 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.257065 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.257163 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for oryx/127.25.127.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.127.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.127.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.188.keytab.
Entry for principal oryx/127.25.127.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.188.keytab.
I20260619 08:23:59.282577 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.188:34461
--webserver_interface=127.25.127.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.188.keytab
--principal=oryx/127.25.127.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:37543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:35483,127.25.127.189:46777,127.25.127.188:34461
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:23:59.367826 389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:59.368074 389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:59.368116 389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:59.369336 389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:23:59.369405 389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:59.369429 389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:59.369451 389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:23:59.369481 389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:23:59.371021 389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:37543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:35483,127.25.127.189:46777,127.25.127.188:34461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.188.keytab
--principal=oryx/127.25.127.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.188:34461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.127.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:59.371306 389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:59.371546 389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:59.376817 394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:59.376807 395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:59.376802 397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:59.376843 389 server_base.cc:1034] running on GCE node
I20260619 08:23:59.377251 389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:59.377585 389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:59.378772 389 hybrid_clock.cc:648] HybridClock initialized: now 1781857439378755 us; error 43 us; skew 500 ppm
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:59.381664 389 init.cc:377] Logged in from keytab as oryx/127.25.127.188@KRBTEST.COM (short username oryx)
I20260619 08:23:59.382140 389 webserver.cc:460] Webserver started at http://127.25.127.188:39135/ using document root <none> and password file <none>
I20260619 08:23:59.382390 389 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:59.382459 389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:59.382623 389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:23:59.383761 389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/data/instance:
uuid: "beb8df1f97bf484b822e808559f5a228"
format_stamp: "Formatted at 2026-06-19 08:23:59 on dist-test-slave-5rhg"
server_key: "cdf825fc99e643f0a85f70983547e3fc"
server_key_iv: "34a7ab7f0131772a2aeac9581df70573"
server_key_version: "encryptionkey@0"
I20260619 08:23:59.384141 389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/wal/instance:
uuid: "beb8df1f97bf484b822e808559f5a228"
format_stamp: "Formatted at 2026-06-19 08:23:59 on dist-test-slave-5rhg"
server_key: "cdf825fc99e643f0a85f70983547e3fc"
server_key_iv: "34a7ab7f0131772a2aeac9581df70573"
server_key_version: "encryptionkey@0"
I20260619 08:23:59.385571 389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:59.386441 404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.386617 389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:23:59.386718 389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/wal
uuid: "beb8df1f97bf484b822e808559f5a228"
format_stamp: "Formatted at 2026-06-19 08:23:59 on dist-test-slave-5rhg"
server_key: "cdf825fc99e643f0a85f70983547e3fc"
server_key_iv: "34a7ab7f0131772a2aeac9581df70573"
server_key_version: "encryptionkey@0"
I20260619 08:23:59.386809 389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:59.403422 389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:59.406015 389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:59.406185 389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:59.411296 389 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.188:34461
I20260619 08:23:59.411329 456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.188:34461 every 8 connection(s)
I20260619 08:23:59.411774 389 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/data/info.pb
I20260619 08:23:59.412904 457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:59.414489 457 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:23:59.417842 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 389
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.188@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.127.188@KRBTEST.COM
I20260619 08:23:59.424783 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.415333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43428 (local address 127.25.127.190:35483)
0619 08:23:59.415404 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.415406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.415535 (+ 129us) server_negotiation.cc:407] Connection header received
0619 08:23:59.416308 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.416309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.416322 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.416374 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.417069 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.417561 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.418946 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.419147 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.419231 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.421695 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.421697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.421698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.421705 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.423320 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.423743 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.423744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.423745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.423810 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.424065 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.424066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.424067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.424247 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.424323 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.424595 (+ 272us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.424689 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 08:23:59.425064 459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.415183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:43428)
0619 08:23:59.415407 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.415456 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.416194 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.416456 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.416460 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.416742 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.416961 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.416965 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.417670 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.417671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.418866 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.418868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.419259 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.419261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.419419 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.419962 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.419972 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.421603 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.423424 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.423433 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.423435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.423664 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.423880 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.423880 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.423881 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.423997 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.424312 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.424319 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.424461 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.424571 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.424746 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":119,"threads_started":1}
I20260619 08:23:59.425753 457 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:23:59.427454 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.418375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39106 (local address 127.25.127.188:34461)
0619 08:23:59.418552 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.418557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.418576 (+ 19us) server_negotiation.cc:407] Connection header received
0619 08:23:59.418597 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.418599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.418627 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.418708 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.419327 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.420348 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.421064 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.421464 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.421546 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.424102 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.424110 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.424110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.424119 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.426190 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.426581 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.426584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.426584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.426644 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.426827 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.426828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.426828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.426992 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.427149 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.427248 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.427322 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.188@KRBTEST.COM for oryx/127.25.127.189@KRBTEST.COM
I20260619 08:23:59.432788 373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.426081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41216 (local address 127.25.127.189:46777)
0619 08:23:59.426138 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.426140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.426174 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:23:59.426223 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.426224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.426237 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.426289 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.426576 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.427345 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.428123 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.428385 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.428473 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.430263 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.430265 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.430266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.430273 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.431638 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.431990 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.431992 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.431992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.432053 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.432287 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.432287 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.432288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.432407 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.432486 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.432616 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.432702 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:23:59.432788 459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.426010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:46777 (local address 127.0.0.1:41216)
0619 08:23:59.426124 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.426141 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.426181 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.426290 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.426291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.426440 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.426497 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.426498 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.427431 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.427432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.428027 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.428029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.428474 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.428475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.428548 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.428980 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.428984 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.430177 (+ 1193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.431737 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.431738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.431739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.431904 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.432169 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.432170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.432170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.432235 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.432475 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.432475 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.432518 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.432587 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.432693 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260619 08:23:59.433563 457 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.188@KRBTEST.COM for oryx/127.25.127.188@KRBTEST.COM
I20260619 08:23:59.439374 459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.433843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:34461 (local address 127.0.0.1:39112)
0619 08:23:59.433891 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.433902 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.433938 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.434035 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.434035 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.434172 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.434199 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.434200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.434827 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.434828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.435297 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.435298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.435693 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.435693 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.435747 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.436091 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.436095 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.437299 (+ 1204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.438563 (+ 1264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.438563 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.438564 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.438726 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.438887 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.438888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.438888 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.438936 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.439131 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.439132 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.439168 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.439218 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.439292 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 08:23:59.439453 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.433906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39112 (local address 127.25.127.188:34461)
0619 08:23:59.433949 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.433951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.433959 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:23:59.433978 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.433978 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.433989 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.434035 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.434277 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.434750 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.435383 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.435602 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.435687 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.437372 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.437373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.437374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.437379 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.438492 (+ 1113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.438787 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.438788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.438788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.438831 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.438986 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.438986 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.438986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.439077 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.439126 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.439232 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.439334 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":27}
I20260619 08:23:59.440861 457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: Bootstrap starting.
I20260619 08:23:59.442000 457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:59.442489 457 log.cc:824] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:59.443336 457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: No bootstrap required, opened a new log
I20260619 08:23:59.445039 457 raft_consensus.cc:348] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } }
I20260619 08:23:59.445171 457 raft_consensus.cc:374] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:59.445195 457 raft_consensus.cc:729] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beb8df1f97bf484b822e808559f5a228, State: Initialized, Role: FOLLOWER
I20260619 08:23:59.445350 457 consensus_queue.cc:260] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } }
I20260619 08:23:59.445663 457 sys_catalog.cc:564] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:59.445639 462 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } } }
I20260619 08:23:59.445744 462 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 08:23:59.447432 473 catalog_manager.cc:1558] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:23:59.447494 473 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:23:59.449960 389 master_runner.cc:428] Detected that this master beb8df1f97bf484b822e808559f5a228 is joining an existing cluster
I20260619 08:23:59.450027 389 master_runner.cc:432] Initiating AddMaster RPC to add 127.25.127.188:34461
I20260619 08:23:59.459012 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.451158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43442 (local address 127.25.127.190:35483)
0619 08:23:59.451224 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.451226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.451463 (+ 237us) server_negotiation.cc:407] Connection header received
0619 08:23:59.451576 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.451577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.451591 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.451661 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.452571 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.453330 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.454488 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.454916 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.455018 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.456297 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.456299 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.456299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.456306 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.457825 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.458192 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.458193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.458193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.458262 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.458465 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.458466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.458466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.458596 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.458669 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.458831 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.458912 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:23:59.459045 479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.451168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:43442)
0619 08:23:59.451393 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.451405 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.451493 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.451759 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.451760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.452371 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.452481 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.452483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.453456 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.453457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.454400 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.454401 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.455095 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.455096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.455156 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.455724 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.455729 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.456134 (+ 405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.457930 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.457931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.457931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.458116 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.458329 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.458330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.458330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.458392 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.458674 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.458675 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.458720 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.458805 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.458914 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":32,"thread_start_us":170,"threads_started":1}
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857438, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.190@KRBTEST.COM for oryx/127.25.127.188@KRBTEST.COM
I20260619 08:23:59.465786 462 raft_consensus.cc:482] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:23:59.465906 462 raft_consensus.cc:504] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } }
I20260619 08:23:59.466200 462 leader_election.cc:288] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a382c3e1cdce42d89abf62d01255925d (127.25.127.190:35483), 473f4c6d5426459a9c73ab26656abe8d (127.25.127.189:46777)
I20260619 08:23:59.466390 32216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "beb8df1f97bf484b822e808559f5a228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a382c3e1cdce42d89abf62d01255925d" is_pre_election: true
I20260619 08:23:59.466400 345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "beb8df1f97bf484b822e808559f5a228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "473f4c6d5426459a9c73ab26656abe8d" is_pre_election: true
I20260619 08:23:59.466542 345 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 LEARNER]: Handling vote request from an unknown peer beb8df1f97bf484b822e808559f5a228
I20260619 08:23:59.466594 345 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate beb8df1f97bf484b822e808559f5a228 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260619 08:23:59.466790 406 leader_election.cc:302] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: beb8df1f97bf484b822e808559f5a228; no voters: 473f4c6d5426459a9c73ab26656abe8d, a382c3e1cdce42d89abf62d01255925d
I20260619 08:23:59.466993 462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:23:59.486807 462 raft_consensus.cc:2726] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 08:23:59.487680 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 32268
I20260619 08:23:59.489198 373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.451158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41220 (local address 127.25.127.189:46777)
0619 08:23:59.451204 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.451206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.451697 (+ 491us) server_negotiation.cc:407] Connection header received
0619 08:23:59.451751 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.451752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.451763 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.451821 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.452440 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.453279 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.454614 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.454915 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.454957 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.485889 (+ 30932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.485891 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.485891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.485900 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.487534 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.488377 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.488378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.488379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.488443 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.488634 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.488634 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.488635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.488771 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.488822 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.489002 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.489100 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:23:59.489557 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.451244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:46777 (local address 127.0.0.1:41220)
0619 08:23:59.451641 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.451651 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.451701 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.451871 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.451872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.452223 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.452306 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.452309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.453442 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.453443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.454509 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.454512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.455036 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.455037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.455150 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.484805 (+ 29655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.484812 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.485751 (+ 939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.487636 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.487637 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.487638 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.488282 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.488512 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.488513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.488513 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.488567 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.488852 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.488853 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.488899 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.489326 (+ 427us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.489446 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":37,"thread_start_us":112,"threads_started":1}
I20260619 08:23:59.491556 32763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.461255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:34461 (local address 127.0.0.1:39124)
0619 08:23:59.461307 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.461318 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.461355 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.461678 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.461679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.461859 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.461907 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.461909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.462702 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.462703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.463572 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.463575 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.463970 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.463971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.464040 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.464416 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.464422 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.465820 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.490254 (+ 24434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.490256 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.490256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.490534 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.490782 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.490783 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.490783 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.490854 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.491136 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.491137 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.491188 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.491237 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.491395 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 08:23:59.492079 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.451574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39118 (local address 127.25.127.188:34461)
0619 08:23:59.451626 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.451628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.451830 (+ 202us) server_negotiation.cc:407] Connection header received
0619 08:23:59.451905 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.451906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.451914 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.451938 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.452520 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.453245 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.454482 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.454875 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.454931 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.485722 (+ 30791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.485724 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.485724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.485731 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.490975 (+ 5244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.491334 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.491335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.491335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.491378 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.491558 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.491559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.491559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.491668 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.491707 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.491884 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.491956 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 08:23:59.492249 482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.461382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39124 (local address 127.25.127.188:34461)
0619 08:23:59.461543 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.461545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.461550 (+ 5us) server_negotiation.cc:407] Connection header received
0619 08:23:59.461580 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.461580 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.461595 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.461642 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.462028 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.462625 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.463649 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.463896 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.463952 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.465983 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.465984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.465985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.465992 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.490167 (+ 24175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.490629 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.490630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.490630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.490689 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.490920 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.490921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.490921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.491062 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.491883 (+ 821us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.491927 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.492021 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260619 08:23:59.493104 480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.451518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:34461 (local address 127.0.0.1:39118)
0619 08:23:59.451784 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.451793 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.451848 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.451991 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.451992 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.452347 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.452417 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.452419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.453324 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.453324 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.454387 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.454390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.455031 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.455032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.455097 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.484978 (+ 29881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.484984 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.485579 (+ 595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.491059 (+ 5480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.491059 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.491060 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.491259 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.491438 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.491438 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.491439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.491485 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.491746 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.491747 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.491792 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.492038 (+ 246us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.492254 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":70,"threads_started":1}
I20260619 08:23:59.493410 32206 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.25.127.188:34461: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "beb8df1f97bf484b822e808559f5a228"
member_type: NON_VOTER
last_known_addr {
host: "127.25.127.188"
port: 34461
}
attrs {
promote: true
}
}
dest_uuid: "a382c3e1cdce42d89abf62d01255925d"
cas_config_opid_index: 5
I20260619 08:23:59.493685 32206 consensus_queue.cc:237] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } }
I20260619 08:23:59.494072 32764 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER beb8df1f97bf484b822e808559f5a228 (127.25.127.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } } }
I20260619 08:23:59.494571 431 raft_consensus.cc:1261] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 1 FOLLOWER]: Refusing update from remote peer a382c3e1cdce42d89abf62d01255925d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 08:23:59.495002 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:41471
--local_ip_for_outbound_sockets=127.25.127.129
--tserver_master_addrs=127.25.127.190:35483
--webserver_port=33625
--webserver_interface=127.25.127.129
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.129.keytab
--principal=oryx/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:37543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.127.190:35483,127.25.127.189:46777
--tserver_master_addrs=127.25.127.190:35483,127.25.127.189:46777,127.25.127.188:34461 with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:23:59.495330 32764 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.25.127.188:34461
I20260619 08:23:59.495404 32764 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [LEADER]: Connected to new peer: Peer: permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:23:59.495664 32764 sys_catalog.cc:455] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } } }
I20260619 08:23:59.495739 32764 sys_catalog.cc:458] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:59.496649 389 master.cc:537] Master@127.25.127.188:34461 shutting down...
I20260619 08:23:59.497272 431 raft_consensus.cc:858] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } }
W20260619 08:23:59.497346 431 raft_consensus.cc:1565] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } }
I20260619 08:23:59.498536 32763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.494163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:46777 (local address 127.0.0.1:41234)
0619 08:23:59.494800 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.494812 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.494849 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.495091 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.495092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.495093 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:59.495141 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.495143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.496190 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.496191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.497387 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.497391 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.498272 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.498273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.498342 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.498371 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.498423 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.498468 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620}
I20260619 08:23:59.498816 373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.494291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41234 (local address 127.25.127.189:46777)
0619 08:23:59.494922 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.494924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.494931 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:23:59.494951 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.494952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.494973 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.495027 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:59.495211 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.496055 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.497497 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.498087 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.498158 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.498237 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.498703 (+ 466us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.498753 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606}
I20260619 08:23:59.499001 345 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 LEARNER]: Refusing update from remote peer a382c3e1cdce42d89abf62d01255925d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 08:23:59.499261 32765 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [LEADER]: Connected to new peer: Peer: permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 08:23:59.499331 431 raft_consensus.cc:2932] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 473f4c6d5426459a9c73ab26656abe8d (127.25.127.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } } }
I20260619 08:23:59.499827 345 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER beb8df1f97bf484b822e808559f5a228 (127.25.127.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } } }
I20260619 08:23:59.500211 485 mvcc.cc:205] Tried to move back new op lower bound from 7298488067956789248 to 7298488067856576512. Current Snapshot: MvccSnapshot[applied={T|T < 7298488067866992640}]
I20260619 08:23:59.501032 375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a382c3e1cdce42d89abf62d01255925d. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } } }
I20260619 08:23:59.501127 375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [sys.catalog]: This master's current role is: LEARNER
I20260619 08:23:59.501169 375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } } }
I20260619 08:23:59.501212 375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [sys.catalog]: This master's current role is: LEARNER
I20260619 08:23:59.501224 462 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a382c3e1cdce42d89abf62d01255925d. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } } }
I20260619 08:23:59.501297 462 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260619 08:23:59.501332 462 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } } }
I20260619 08:23:59.501379 462 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260619 08:23:59.501420 462 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } } }
I20260619 08:23:59.501577 462 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260619 08:23:59.501708 389 raft_consensus.cc:2229] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260619 08:23:59.501706 32765 sys_catalog.cc:455] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: NON_VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: true } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } } }
I20260619 08:23:59.501780 389 raft_consensus.cc:2258] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260619 08:23:59.501786 32765 sys_catalog.cc:458] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:59.501811 389 tablet_replica.cc:330] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: stopping tablet replica
W20260619 08:23:59.502307 32190 consensus_peers.cc:487] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d -> Peer beb8df1f97bf484b822e808559f5a228 (127.25.127.188:34461): Couldn't send request to peer beb8df1f97bf484b822e808559f5a228. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 08:23:59.502938 483 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d: attempting to promote NON_VOTER 473f4c6d5426459a9c73ab26656abe8d to VOTER
I20260619 08:23:59.503160 483 consensus_queue.cc:237] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } }
I20260619 08:23:59.503974 345 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 LEARNER]: Refusing update from remote peer a382c3e1cdce42d89abf62d01255925d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 08:23:59.504212 32764 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [LEADER]: Connected to new peer: Peer: permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260619 08:23:59.504936 32190 consensus_peers.cc:487] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d -> Peer beb8df1f97bf484b822e808559f5a228 (127.25.127.188:34461): Couldn't send request to peer beb8df1f97bf484b822e808559f5a228. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 08:23:59.505046 32764 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 473f4c6d5426459a9c73ab26656abe8d (127.25.127.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } } }
I20260619 08:23:59.506122 32764 sys_catalog.cc:455] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } } }
I20260619 08:23:59.506222 32764 sys_catalog.cc:458] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:59.506286 32764 sys_catalog.cc:455] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } } }
I20260619 08:23:59.506353 32764 sys_catalog.cc:458] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:59.506644 345 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 473f4c6d5426459a9c73ab26656abe8d (127.25.127.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } } }
I20260619 08:23:59.507689 493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } } }
I20260619 08:23:59.507794 493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:23:59.520944 389 master.cc:552] Master@127.25.127.188:34461 shutdown complete.
I20260619 08:23:59.521014 389 master_runner.cc:305] Clearing existing system tablet
I20260619 08:23:59.521157 389 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:59.521209 389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:59.522006 389 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:23:59.522599 495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.522753 389 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:59.522823 389 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:23:59.522851 389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/wal
uuid: "beb8df1f97bf484b822e808559f5a228"
format_stamp: "Formatted at 2026-06-19 08:23:59 on dist-test-slave-5rhg"
server_key: "cdf825fc99e643f0a85f70983547e3fc"
server_key_iv: "34a7ab7f0131772a2aeac9581df70573"
server_key_version: "encryptionkey@0"
I20260619 08:23:59.523579 389 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:23:59.525161 389 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 08:23:59.525238 389 log.cc:1196] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 08:23:59.525501 389 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: Deleting consensus metadata
I20260619 08:23:59.525640 389 master_runner.cc:315] Copying system tablet from 127.25.127.190:35483
I20260619 08:23:59.526270 389 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: tablet copy: Beginning tablet copy session from remote peer at address 127.25.127.190:35483
I20260619 08:23:59.533999 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.526580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:43446)
0619 08:23:59.526704 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.526714 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.526775 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.526937 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.526937 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.527151 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.527214 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.527217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.528319 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.528320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.529240 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.529244 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.529714 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.529715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.529777 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.530321 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.530326 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.530750 (+ 424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.532711 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.532712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.532713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.532949 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.533251 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.533252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.533252 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.533327 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.533641 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.533642 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.533701 (+ 59us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.533775 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.533887 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260619 08:23:59.533996 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.526665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43446 (local address 127.25.127.190:35483)
0619 08:23:59.526715 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.526717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.526763 (+ 46us) server_negotiation.cc:407] Connection header received
0619 08:23:59.526833 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.526834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.526846 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.526923 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.527303 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.528226 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.529341 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.529631 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.529706 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.530851 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.530853 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.530853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.530861 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.532623 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.533069 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.533071 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.533071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.533169 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.533410 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.533411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.533411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.533562 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.533636 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.533821 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.533909 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:23:59.534763 32226 tablet_copy_service.cc:140] P a382c3e1cdce42d89abf62d01255925d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer beb8df1f97bf484b822e808559f5a228 ({username='oryx', principal='oryx/127.25.127.188@KRBTEST.COM'} at 127.0.0.1:43446)
I20260619 08:23:59.534833 32226 tablet_copy_service.cc:161] P a382c3e1cdce42d89abf62d01255925d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer beb8df1f97bf484b822e808559f5a228 at {username='oryx', principal='oryx/127.25.127.188@KRBTEST.COM'} at 127.0.0.1:43446: session id = beb8df1f97bf484b822e808559f5a228-00000000000000000000000000000000
I20260619 08:23:59.535456 32226 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 08:23:59.535952 389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 08:23:59.537348 389 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: tablet copy: Starting download of 0 data blocks...
I20260619 08:23:59.537510 389 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: tablet copy: Starting download of 1 WAL segments...
I20260619 08:23:59.538283 389 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 08:23:59.539170 32226 tablet_copy_service.cc:342] P a382c3e1cdce42d89abf62d01255925d: Request end of tablet copy session beb8df1f97bf484b822e808559f5a228-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.25.127.188@KRBTEST.COM'} at 127.0.0.1:43446
I20260619 08:23:59.539230 32226 tablet_copy_service.cc:433] P a382c3e1cdce42d89abf62d01255925d: ending tablet copy session beb8df1f97bf484b822e808559f5a228-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer beb8df1f97bf484b822e808559f5a228
I20260619 08:23:59.541635 389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:59.542999 505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:59.543143 503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:59.543290 389 server_base.cc:1034] running on GCE node
W20260619 08:23:59.543277 502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:59.543438 389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:59.543568 389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:59.544687 389 hybrid_clock.cc:648] HybridClock initialized: now 1781857439544677 us; error 28 us; skew 500 ppm
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:59.546293 389 init.cc:377] Logged in from keytab as oryx/127.25.127.188@KRBTEST.COM (short username oryx)
I20260619 08:23:59.546622 389 webserver.cc:460] Webserver started at http://127.25.127.188:34545/ using document root <none> and password file <none>
I20260619 08:23:59.546741 389 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:59.546774 389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:59.547550 389 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:23:59.548175 512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.548353 389 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.548408 389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/wal
uuid: "beb8df1f97bf484b822e808559f5a228"
format_stamp: "Formatted at 2026-06-19 08:23:59 on dist-test-slave-5rhg"
server_key: "cdf825fc99e643f0a85f70983547e3fc"
server_key_iv: "34a7ab7f0131772a2aeac9581df70573"
server_key_version: "encryptionkey@0"
I20260619 08:23:59.548470 389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:59.557564 389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:59.559587 389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:59.562606 389 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.188:34461
I20260619 08:23:59.563019 389 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/master-2/data/info.pb
I20260619 08:23:59.564352 564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.188:34461 every 8 connection(s)
I20260619 08:23:59.565234 565 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:23:59.565747 565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: Bootstrap starting.
I20260619 08:23:59.568042 565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:23:59.568284 565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: Bootstrap complete.
I20260619 08:23:59.568492 565 raft_consensus.cc:348] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } }
I20260619 08:23:59.568579 565 raft_consensus.cc:729] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: beb8df1f97bf484b822e808559f5a228, State: Initialized, Role: LEARNER
I20260619 08:23:59.568624 565 consensus_queue.cc:260] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } }
I20260619 08:23:59.568818 565 sys_catalog.cc:564] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:23:59.568806 568 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } } }
I20260619 08:23:59.569036 568 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: This master's current role is: LEARNER
I20260619 08:23:59.570276 579 catalog_manager.cc:1259] Loaded cluster ID: 5f7f4708a0c94aab8e7bf11e4f6f51f7
I20260619 08:23:59.570370 579 catalog_manager.cc:1552] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: loading cluster ID for follower catalog manager: success
I20260619 08:23:59.572237 579 catalog_manager.cc:1574] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: acquiring CA information for follower catalog manager: success
I20260619 08:23:59.572644 579 catalog_manager.cc:1602] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 08:23:59.601042 484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:59.601250 484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:59.601274 484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:59.602444 484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:59.602485 484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:59.602514 484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:23:59.604138 484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:37543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.129.keytab
--principal=oryx/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:41471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=33625
--tserver_master_addrs=127.25.127.190:35483,127.25.127.189:46777,127.25.127.188:34461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:59.604427 484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:59.604671 484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:59.609788 584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:59.609861 585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:59.609861 587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:59.610100 484 server_base.cc:1034] running on GCE node
I20260619 08:23:59.610265 484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:59.610499 484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:59.611639 484 hybrid_clock.cc:648] HybridClock initialized: now 1781857439611614 us; error 39 us; skew 500 ppm
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:59.614305 484 init.cc:377] Logged in from keytab as oryx/127.25.127.129@KRBTEST.COM (short username oryx)
I20260619 08:23:59.614773 484 webserver.cc:460] Webserver started at http://127.25.127.129:33625/ using document root <none> and password file <none>
I20260619 08:23:59.615018 484 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:59.615109 484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:59.616496 484 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:23:59.617134 594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.617328 484 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:23:59.617434 484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "e891c0b19a604149b7038f924fb83195"
format_stamp: "Formatted at 2026-06-19 08:23:58 on dist-test-slave-5rhg"
server_key: "2dd25c02ae98de3258a2b74a3da3d9a7"
server_key_iv: "05fecc61260170525fa9443bf993df11"
server_key_version: "encryptionkey@0"
I20260619 08:23:59.617766 484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:59.640825 484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:59.643607 484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:59.643777 484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:59.644022 484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:59.644373 484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:59.644409 484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.644433 484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:59.644480 484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.652510 484 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:41471
I20260619 08:23:59.652587 707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:41471 every 8 connection(s)
I20260619 08:23:59.652989 484 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:23:59.656203 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 484
I20260619 08:23:59.656358 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 32402
I20260619 08:23:59.662936 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:34349
--local_ip_for_outbound_sockets=127.25.127.130
--tserver_master_addrs=127.25.127.190:35483
--webserver_port=36279
--webserver_interface=127.25.127.130
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.130.keytab
--principal=oryx/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:37543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.127.190:35483,127.25.127.189:46777
--tserver_master_addrs=127.25.127.190:35483,127.25.127.189:46777,127.25.127.188:34461 with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.188@KRBTEST.COM
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.189@KRBTEST.COM
I20260619 08:23:59.670845 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.654278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:56589 (local address 127.25.127.190:35483)
0619 08:23:59.654328 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.654330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.654524 (+ 194us) server_negotiation.cc:407] Connection header received
0619 08:23:59.655345 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.655346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.655360 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.655394 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.656366 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.657145 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.658784 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.659103 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.659142 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.667238 (+ 8096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.667241 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.667242 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.667250 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.669216 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.669886 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.669887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.669888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.669946 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.670174 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.670175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.670175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.670321 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.670377 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.670614 (+ 237us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.670717 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:23:59.670979 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.653706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:35119 (local address 127.25.127.188:34461)
0619 08:23:59.653880 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.653884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.654062 (+ 178us) server_negotiation.cc:407] Connection header received
0619 08:23:59.655328 (+ 1266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.655329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.655346 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.655411 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.656423 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.657498 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.658771 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.659124 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.659188 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.666720 (+ 7532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.666722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.666723 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.666734 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.668701 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.669384 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.669386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.669386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.669452 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.669772 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.669772 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.669773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.669968 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.670034 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.670727 (+ 693us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.670836 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
I20260619 08:23:59.671499 712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.653666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:34461 (local address 127.25.127.129:35119)
0619 08:23:59.653824 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.653872 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.655205 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.655506 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.655515 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.655926 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.656291 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.656295 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.657602 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.657603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.658676 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.658679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.659243 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.659244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.659393 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.664034 (+ 4641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.664038 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.666560 (+ 2522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.668834 (+ 2274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.668843 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.668846 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.669266 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.669531 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.669532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.669532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.669690 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.670109 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.670116 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.670318 (+ 202us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.670473 (+ 155us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.671195 (+ 722us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":8,"thread_start_us":61,"threads_started":1}
I20260619 08:23:59.671497 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.654217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.129:56589)
0619 08:23:59.654481 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.654488 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.655220 (+ 732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.655506 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.655511 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.655929 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.656252 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.656258 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.657266 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.657267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.658675 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.658677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.659237 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.659238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.659394 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.663969 (+ 4575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.663984 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.666907 (+ 2923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.669307 (+ 2400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.669308 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.669308 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.669787 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.670023 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.670024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.670024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.670086 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.670423 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.670424 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.670469 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.670977 (+ 508us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.671271 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":186,"thread_start_us":60,"threads_started":1}
I20260619 08:23:59.672008 708 heartbeater.cc:347] Connected to a master server at 127.25.127.188:34461
I20260619 08:23:59.672037 709 heartbeater.cc:347] Connected to a master server at 127.25.127.190:35483
I20260619 08:23:59.672097 708 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:59.672099 709 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:59.672327 708 heartbeater.cc:511] Master 127.25.127.188:34461 requested a full tablet report, sending...
I20260619 08:23:59.672328 709 heartbeater.cc:511] Master 127.25.127.190:35483 requested a full tablet report, sending...
I20260619 08:23:59.672751 32206 ts_manager.cc:194] Re-registered known tserver with Master: e891c0b19a604149b7038f924fb83195 (127.25.127.129:41471)
I20260619 08:23:59.672851 524 ts_manager.cc:194] Registered new tserver with Master: e891c0b19a604149b7038f924fb83195 (127.25.127.129:41471)
I20260619 08:23:59.673326 32206 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:56589
I20260619 08:23:59.673703 373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.654396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:60545 (local address 127.25.127.189:46777)
0619 08:23:59.654443 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.654445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.654454 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:23:59.655344 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.655346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.655360 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.655394 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.656365 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.657138 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.658784 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.659098 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.659142 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.668616 (+ 9474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.668618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.668619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.668627 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.670300 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.670823 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.670824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.670824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.670880 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.673140 (+ 2260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.673141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.673141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.673285 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.673320 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.673529 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.673628 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:23:59.674081 715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.654080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:46777 (local address 127.25.127.129:60545)
0619 08:23:59.654387 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.654396 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.655220 (+ 824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.655506 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.655511 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.655930 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.656252 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.656258 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.657472 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.657473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.658675 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.658677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.659236 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.659237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.659394 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.664300 (+ 4906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.664306 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.668489 (+ 4183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.670416 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.670417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.670417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.670739 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.672991 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.672991 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.672992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.673060 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.673358 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.673359 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.673416 (+ 57us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.673841 (+ 425us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.674010 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":90,"thread_start_us":113,"threads_started":1}
I20260619 08:23:59.674693 710 heartbeater.cc:347] Connected to a master server at 127.25.127.189:46777
I20260619 08:23:59.674750 710 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:59.674813 710 heartbeater.cc:511] Master 127.25.127.189:46777 requested a full tablet report, sending...
I20260619 08:23:59.675186 335 ts_manager.cc:194] Registered new tserver with Master: e891c0b19a604149b7038f924fb83195 (127.25.127.129:41471)
W20260619 08:23:59.751087 716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:59.751354 716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:59.751384 716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:59.752825 716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:59.752871 716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:59.752897 716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:23:59.754536 716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:37543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.130.keytab
--principal=oryx/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:34349
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=36279
--tserver_master_addrs=127.25.127.190:35483,127.25.127.189:46777,127.25.127.188:34461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:59.754725 716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:59.754899 716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:23:59.759819 722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:59.759795 721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:59.759807 724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:59.760267 716 server_base.cc:1034] running on GCE node
I20260619 08:23:59.760599 716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:59.760826 716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:59.761966 716 hybrid_clock.cc:648] HybridClock initialized: now 1781857439761954 us; error 29 us; skew 500 ppm
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:59.764703 716 init.cc:377] Logged in from keytab as oryx/127.25.127.130@KRBTEST.COM (short username oryx)
I20260619 08:23:59.765092 716 webserver.cc:460] Webserver started at http://127.25.127.130:36279/ using document root <none> and password file <none>
I20260619 08:23:59.765280 716 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:59.765331 716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:59.766589 716 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:23:59.767118 731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.767288 716 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.767390 716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "15f897dd709d42f48d6b51b65720e9bc"
format_stamp: "Formatted at 2026-06-19 08:23:58 on dist-test-slave-5rhg"
server_key: "69697cf450e06729cc17f3007534cfc0"
server_key_iv: "29a8531055af6d6af580c8c2f34fbe94"
server_key_version: "encryptionkey@0"
I20260619 08:23:59.767743 716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:59.784839 716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:59.787271 716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:59.787446 716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:59.787703 716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:59.788020 716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:59.788072 716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.788108 716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:59.788127 716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.794085 716 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:34349
I20260619 08:23:59.794147 844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:34349 every 8 connection(s)
I20260619 08:23:59.794476 716 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:23:59.798484 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 716
I20260619 08:23:59.798648 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 32536
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.188@KRBTEST.COM
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.189@KRBTEST.COM
I20260619 08:23:59.804800 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:39721
--local_ip_for_outbound_sockets=127.25.127.131
--tserver_master_addrs=127.25.127.190:35483
--webserver_port=37765
--webserver_interface=127.25.127.131
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.131.keytab
--principal=oryx/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:37543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.127.190:35483,127.25.127.189:46777
--tserver_master_addrs=127.25.127.190:35483,127.25.127.189:46777,127.25.127.188:34461 with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM
I20260619 08:23:59.806156 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.795349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:51315 (local address 127.25.127.188:34461)
0619 08:23:59.795424 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.795426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.795713 (+ 287us) server_negotiation.cc:407] Connection header received
0619 08:23:59.796248 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.796249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.796267 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.796318 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.797115 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.797869 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.799010 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.799294 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.799333 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.802643 (+ 3310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.802644 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.802648 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.802656 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.804487 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.805042 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.805044 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.805044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.805105 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.805412 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.805413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.805413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.805558 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.805611 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.805958 (+ 347us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.806054 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 08:23:59.806353 851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.795207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:34461 (local address 127.25.127.130:51315)
0619 08:23:59.795657 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.795667 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.796131 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.796764 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.796765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.796950 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.796996 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.796997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.797975 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.797975 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.798920 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.798923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.799417 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.799418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.799547 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.800378 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.800388 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.802533 (+ 2145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.804596 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.804612 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.804614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.804946 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.805181 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.805182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.805182 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.805317 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.805651 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.805656 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.805818 (+ 162us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.805936 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.806091 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":240,"thread_start_us":147,"threads_started":1}
I20260619 08:23:59.807039 845 heartbeater.cc:347] Connected to a master server at 127.25.127.188:34461
I20260619 08:23:59.807140 845 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:59.807333 845 heartbeater.cc:511] Master 127.25.127.188:34461 requested a full tablet report, sending...
I20260619 08:23:59.807682 849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.795207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:46777 (local address 127.25.127.130:55185)
0619 08:23:59.795585 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.795621 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.796126 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.796426 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.796432 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.796836 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.796991 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.796997 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.797975 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.797975 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.800049 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.800051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.800621 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.800622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.800680 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.801089 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.801094 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.803159 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.806467 (+ 3308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.806468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.806469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.806704 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.806951 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.806952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.806952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.807016 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.807312 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.807312 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.807352 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.807436 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.807551 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":106,"thread_start_us":144,"threads_started":1}
I20260619 08:23:59.807958 524 ts_manager.cc:194] Registered new tserver with Master: 15f897dd709d42f48d6b51b65720e9bc (127.25.127.130:34349)
I20260619 08:23:59.808331 373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.795366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:55185 (local address 127.25.127.189:46777)
0619 08:23:59.795412 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.795414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.795703 (+ 289us) server_negotiation.cc:407] Connection header received
0619 08:23:59.796252 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.796253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.796267 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.796299 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.797104 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.797872 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.800165 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.800479 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.800527 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.803249 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.803250 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.803251 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.803259 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.805227 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.806811 (+ 1584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.806817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.806817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.806871 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.807087 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.807087 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.807088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.807221 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.807285 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.808109 (+ 824us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.808237 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:23:59.808663 847 heartbeater.cc:347] Connected to a master server at 127.25.127.189:46777
I20260619 08:23:59.808744 847 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:59.808830 847 heartbeater.cc:511] Master 127.25.127.189:46777 requested a full tablet report, sending...
I20260619 08:23:59.809132 335 ts_manager.cc:194] Registered new tserver with Master: 15f897dd709d42f48d6b51b65720e9bc (127.25.127.130:34349)
I20260619 08:23:59.810030 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.795327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:60637 (local address 127.25.127.190:35483)
0619 08:23:59.795380 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.795383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.795687 (+ 304us) server_negotiation.cc:407] Connection header received
0619 08:23:59.796369 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.796370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.796380 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.796446 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.797086 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.797607 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.799235 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.799571 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.799624 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.806144 (+ 6520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.806146 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.806146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.806155 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.808237 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.808724 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.808725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.808726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.808813 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.809397 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.809399 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.809399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.809551 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.809639 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.809810 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.809897 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:23:59.810513 850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.795207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.130:60637)
0619 08:23:59.795554 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.795613 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.796105 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.796451 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.796453 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.796795 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.796990 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.796995 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.797714 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.797715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.799136 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.799139 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.799705 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.799706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.799780 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.804497 (+ 4717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.804503 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.806033 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.808341 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.808341 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.808342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.808668 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.808888 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.808889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.808889 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.809322 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.809651 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.809652 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.809693 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.809757 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.809829 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":130,"threads_started":1}
I20260619 08:23:59.810913 846 heartbeater.cc:347] Connected to a master server at 127.25.127.190:35483
I20260619 08:23:59.810986 846 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:59.811059 846 heartbeater.cc:511] Master 127.25.127.190:35483 requested a full tablet report, sending...
I20260619 08:23:59.811316 32206 ts_manager.cc:194] Re-registered known tserver with Master: 15f897dd709d42f48d6b51b65720e9bc (127.25.127.130:34349)
I20260619 08:23:59.811882 32206 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:60637
W20260619 08:23:59.890025 852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:23:59.890228 852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:23:59.890251 852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:23:59.891355 852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:23:59.891392 852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:23:59.891417 852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:23:59.892933 852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:37543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.131.keytab
--principal=oryx/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:39721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=37765
--tserver_master_addrs=127.25.127.190:35483,127.25.127.189:46777,127.25.127.188:34461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:23:59.893187 852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:23:59.893430 852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:23:59.895895 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.893085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39126 (local address 127.25.127.188:34461)
0619 08:23:59.893137 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.893139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.893156 (+ 17us) server_negotiation.cc:407] Connection header received
0619 08:23:59.893242 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.893243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.893272 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.893349 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:59.893537 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.894047 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.895145 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.895570 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.895661 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.895737 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.895786 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.895826 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 08:23:59.895896 32763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.893040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:34461 (local address 127.0.0.1:39126)
0619 08:23:59.893105 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.893117 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.893174 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.893356 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.893357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.893358 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:59.893417 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.893419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.894144 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.894145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.895050 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.895053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.895661 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.895662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.895725 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.895729 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.895781 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.895831 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 08:23:59.897639 539 raft_consensus.cc:1203] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260619 08:23:59.897889 568 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a382c3e1cdce42d89abf62d01255925d. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } } }
I20260619 08:23:59.898007 568 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: This master's current role is: LEARNER
W20260619 08:23:59.898451 858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:59.898418 857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:23:59.898433 860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:23:59.898777 852 server_base.cc:1034] running on GCE node
I20260619 08:23:59.898948 852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:23:59.899147 852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:23:59.900283 852 hybrid_clock.cc:648] HybridClock initialized: now 1781857439900263 us; error 36 us; skew 500 ppm
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:23:59.902647 852 init.cc:377] Logged in from keytab as oryx/127.25.127.131@KRBTEST.COM (short username oryx)
I20260619 08:23:59.903133 852 webserver.cc:460] Webserver started at http://127.25.127.131:37765/ using document root <none> and password file <none>
I20260619 08:23:59.903362 852 fs_manager.cc:359] Metadata directory not provided
I20260619 08:23:59.903443 852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:23:59.904855 852 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:23:59.905447 867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.905622 852 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:23:59.905728 852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "2d9893d7478843c3b96c5b842fc4d11f"
format_stamp: "Formatted at 2026-06-19 08:23:58 on dist-test-slave-5rhg"
server_key: "b5ee6e3f85023a9c458c51062f915c5d"
server_key_iv: "80f384a24daf1053ae1cc089994fdc89"
server_key_version: "encryptionkey@0"
I20260619 08:23:59.906075 852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:23:59.917438 852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:23:59.919562 852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:23:59.919731 852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:23:59.919997 852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:23:59.920400 852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:23:59.920457 852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.920498 852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:23:59.920542 852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:23:59.926827 852 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:39721
I20260619 08:23:59.926877 980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:39721 every 8 connection(s)
I20260619 08:23:59.927237 852 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:23:59.931733 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 852
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.189@KRBTEST.COM
Jun 19 08:23:59 dist-test-slave-5rhg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.188@KRBTEST.COM
I20260619 08:23:59.940563 373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.928001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:46963 (local address 127.25.127.189:46777)
0619 08:23:59.928047 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.928049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.928345 (+ 296us) server_negotiation.cc:407] Connection header received
0619 08:23:59.929694 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.929696 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.929709 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.929764 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.930654 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.931433 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.932791 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.933099 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.933142 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.937188 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.937189 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.937190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.937198 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.938940 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.939381 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.939383 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.939383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.939448 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.939721 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.939722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.939722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.939862 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.939929 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.940361 (+ 432us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.940442 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:23:59.941220 985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.928008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:46777 (local address 127.25.127.131:46963)
0619 08:23:59.928215 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.928257 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.929468 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.929805 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.929809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.930323 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.930540 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.930547 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.931559 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.931559 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.932690 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.932693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.933227 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.933227 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.933398 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.934328 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.934333 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.937079 (+ 2746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.939036 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.939045 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.939048 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.939287 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.939516 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.939517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.939517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.939648 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.939958 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.939965 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.940182 (+ 217us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.940718 (+ 536us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.940933 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":94,"thread_start_us":156,"threads_started":1}
I20260619 08:23:59.941596 986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.928010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.131:36313)
0619 08:23:59.928405 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.928413 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.929463 (+ 1050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.929805 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.929809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.930319 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.930539 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.930545 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.931295 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.931296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.932686 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.932689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.933222 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.933223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.933399 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.934311 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.934322 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.937927 (+ 3605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.940443 (+ 2516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.940444 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.940444 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.940697 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.940896 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.940897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.940897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.940968 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.941267 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.941268 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.941310 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.941363 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.941489 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":64,"thread_start_us":114,"threads_started":1}
I20260619 08:23:59.942157 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.928031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:34461 (local address 127.25.127.131:49719)
0619 08:23:59.928557 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.928569 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.929480 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.929915 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.929916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.930333 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:23:59.930540 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.930553 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.931529 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.931530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.932712 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.932714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.933225 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.933226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.933398 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.934437 (+ 1039us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:23:59.934441 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:23:59.938653 (+ 4212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:23:59.940528 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.940529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.940529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.941050 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.941259 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:23:59.941259 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:23:59.941260 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:23:59.941315 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:23:59.941613 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:23:59.941618 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:23:59.941673 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.941767 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.941866 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":72,"thread_start_us":105,"threads_started":1}
I20260619 08:23:59.942196 983 heartbeater.cc:347] Connected to a master server at 127.25.127.189:46777
I20260619 08:23:59.942265 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.928337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:49719 (local address 127.25.127.188:34461)
0619 08:23:59.928409 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.928412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.928617 (+ 205us) server_negotiation.cc:407] Connection header received
0619 08:23:59.929833 (+ 1216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.929834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.929851 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.929897 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.930653 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.931429 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.932799 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.933099 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.933142 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.938765 (+ 5623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.938767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.938767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.938775 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.940448 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.941131 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.941133 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.941133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.941191 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.941387 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.941388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.941388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.941518 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.941571 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.942046 (+ 475us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.942152 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:23:59.942296 983 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:59.942510 983 heartbeater.cc:511] Master 127.25.127.189:46777 requested a full tablet report, sending...
I20260619 08:23:59.942520 981 heartbeater.cc:347] Connected to a master server at 127.25.127.188:34461
I20260619 08:23:59.942586 981 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:59.942662 981 heartbeater.cc:511] Master 127.25.127.188:34461 requested a full tablet report, sending...
I20260619 08:23:59.942981 524 ts_manager.cc:194] Registered new tserver with Master: 2d9893d7478843c3b96c5b842fc4d11f (127.25.127.131:39721)
I20260619 08:23:59.943107 335 ts_manager.cc:194] Registered new tserver with Master: 2d9893d7478843c3b96c5b842fc4d11f (127.25.127.131:39721)
I20260619 08:23:59.943897 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.928350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:36313 (local address 127.25.127.190:35483)
0619 08:23:59.928424 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.928426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.928462 (+ 36us) server_negotiation.cc:407] Connection header received
0619 08:23:59.929606 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.929608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.929622 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.929700 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.930652 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.931185 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.932780 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.933080 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.933154 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.938014 (+ 4860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.938016 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.938016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.938024 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.940355 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.940780 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.940781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.940781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.940839 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.941056 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.941056 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.941057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.941182 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.941241 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.943690 (+ 2449us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.943776 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 08:23:59.944286 982 heartbeater.cc:347] Connected to a master server at 127.25.127.190:35483
I20260619 08:23:59.944344 982 heartbeater.cc:464] Registering TS with master...
I20260619 08:23:59.944416 982 heartbeater.cc:511] Master 127.25.127.190:35483 requested a full tablet report, sending...
I20260619 08:23:59.944669 32206 ts_manager.cc:194] Re-registered known tserver with Master: 2d9893d7478843c3b96c5b842fc4d11f (127.25.127.131:39721)
I20260619 08:23:59.945194 32206 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:36313
I20260619 08:23:59.947091 993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.933320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41240 (local address 127.25.127.189:46777)
0619 08:23:59.934170 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.934172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.934179 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:23:59.934200 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.934201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.934212 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.934288 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.935122 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.935915 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.937328 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.937651 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.937718 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.943807 (+ 6089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.943808 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.943808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.943816 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.945814 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.946223 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.946224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.946224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.946280 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.946484 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.946485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.946485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.946608 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.946722 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.946819 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.946905 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":829,"thread_start_us":52,"threads_started":1}
I20260619 08:23:59.948475 992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.932964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39140 (local address 127.25.127.188:34461)
0619 08:23:59.936050 (+ 3086us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.936053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.936061 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:23:59.936087 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.936088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.936100 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.936193 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.937844 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.938636 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.941855 (+ 3219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.942235 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.942280 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.945120 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.945121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.945122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.945129 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.947098 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.947527 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.947528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.947529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.947597 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.947839 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.947840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.947841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.947987 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.948083 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.948270 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.948364 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3060,"thread_start_us":51,"threads_started":1}
I20260619 08:23:59.948535 994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.933567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43456 (local address 127.25.127.190:35483)
0619 08:23:59.933760 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.933762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.935247 (+ 1485us) server_negotiation.cc:407] Connection header received
0619 08:23:59.935523 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.935524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.935539 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.935613 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:23:59.936289 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.937064 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.940544 (+ 3480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.940942 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.941011 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.945173 (+ 4162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:23:59.945174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:23:59.945175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:23:59.945182 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:23:59.947327 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.947718 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.947718 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.947719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.947776 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:23:59.947965 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:23:59.947966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:23:59.947966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:23:59.948109 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:23:59.948243 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.948335 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.948424 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":107,"threads_started":1}
I20260619 08:23:59.950042 32206 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:23:59.951699 32206 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:23:59.952617 32765 sys_catalog.cc:455] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: NON_VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: true } } }
I20260619 08:23:59.952718 32765 sys_catalog.cc:458] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:59.954147 32764 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d: attempting to promote NON_VOTER beb8df1f97bf484b822e808559f5a228 to VOTER
I20260619 08:23:59.954272 32764 consensus_queue.cc:237] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } }
I20260619 08:23:59.954746 539 raft_consensus.cc:1261] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 1 LEARNER]: Refusing update from remote peer a382c3e1cdce42d89abf62d01255925d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260619 08:23:59.954957 345 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 FOLLOWER]: Refusing update from remote peer a382c3e1cdce42d89abf62d01255925d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260619 08:23:59.954936 483 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [LEADER]: Connected to new peer: Peer: permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260619 08:23:59.955160 483 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [LEADER]: Connected to new peer: Peer: permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260619 08:23:59.955801 32764 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, beb8df1f97bf484b822e808559f5a228 (127.25.127.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } } }
I20260619 08:23:59.955950 345 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, beb8df1f97bf484b822e808559f5a228 (127.25.127.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } } }
I20260619 08:23:59.956883 32764 sys_catalog.cc:455] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } } }
I20260619 08:23:59.956911 32765 sys_catalog.cc:455] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } } }
I20260619 08:23:59.957016 32765 sys_catalog.cc:458] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:59.957038 493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } } }
I20260619 08:23:59.957108 493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:23:59.957063 32765 sys_catalog.cc:455] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } } }
I20260619 08:23:59.957136 32765 sys_catalog.cc:458] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:59.957149 539 raft_consensus.cc:2932] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, beb8df1f97bf484b822e808559f5a228 (127.25.127.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } } }
I20260619 08:23:59.958029 568 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a382c3e1cdce42d89abf62d01255925d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } } }
I20260619 08:23:59.958130 568 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:23:59.956972 32764 sys_catalog.cc:458] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d [sys.catalog]: This master's current role is: LEADER
I20260619 08:23:59.962922 1006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.958837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:34349 (local address 127.0.0.1:45560)
0619 08:23:59.959069 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.959078 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.959124 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.959529 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.959544 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.959546 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:59.959617 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.959619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.961053 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.961054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.961786 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.961788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.962639 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.962640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.962709 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.962714 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.962775 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.962823 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":2,"thread_start_us":88,"threads_started":1}
I20260619 08:23:59.962944 1008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.959045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45560 (local address 127.25.127.130:34349)
0619 08:23:59.959318 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.959321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.959329 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:23:59.959400 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.959402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.959420 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.959695 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:59.959759 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.960953 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.961881 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.962553 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.962659 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.962718 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.962790 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.962831 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
I20260619 08:23:59.962925 32763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.958799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:41471 (local address 127.0.0.1:38812)
0619 08:23:59.958850 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.958860 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.958902 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.959477 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.959478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.959480 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:59.959532 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.959535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.960684 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.960685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.961776 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.961779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.962734 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.962735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.962793 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.962795 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.962822 (+ 27us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.962854 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 08:23:59.963176 1005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.958810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:39721 (local address 127.0.0.1:42990)
0619 08:23:59.958937 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 08:23:59.958948 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:23:59.959000 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:23:59.959651 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:23:59.959652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:23:59.959654 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:23:59.959687 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.959688 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.961006 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.961006 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.962082 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:23:59.962085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.962939 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.962940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:23:59.963008 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.963013 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:23:59.963047 (+ 34us) client_negotiation.cc:240] Negotiation successful
0619 08:23:59.963095 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":87,"threads_started":1}
I20260619 08:23:59.963547 1009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.959267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42990 (local address 127.25.127.131:39721)
0619 08:23:59.959462 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.959466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.959474 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:23:59.959559 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.959561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.959576 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.959661 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:59.959730 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.960913 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.962170 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.962845 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.963344 (+ 499us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.963409 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.963433 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.963474 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":100,"threads_started":1}
I20260619 08:23:59.963661 1007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:23:59.959048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38812 (local address 127.25.127.129:41471)
0619 08:23:59.959226 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 08:23:59.959232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:23:59.959244 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:23:59.959336 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:23:59.959339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:23:59.959358 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:23:59.959478 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:23:59.959597 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.960581 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.961898 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:23:59.962648 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:23:59.963384 (+ 736us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:23:59.963450 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:23:59.963476 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 08:23:59.963517 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":107,"threads_started":1}
I20260619 08:23:59.963716 779 tablet_service.cc:1467] Processing CreateTablet for tablet 93d290968bfd4b0e9bd37066b18d4517 (DEFAULT_TABLE table=test-table [id=9367b904e5b74f4ab618d6c1c8aad148]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:59.964097 779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93d290968bfd4b0e9bd37066b18d4517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:59.964130 915 tablet_service.cc:1467] Processing CreateTablet for tablet 93d290968bfd4b0e9bd37066b18d4517 (DEFAULT_TABLE table=test-table [id=9367b904e5b74f4ab618d6c1c8aad148]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:59.964406 642 tablet_service.cc:1467] Processing CreateTablet for tablet 93d290968bfd4b0e9bd37066b18d4517 (DEFAULT_TABLE table=test-table [id=9367b904e5b74f4ab618d6c1c8aad148]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:23:59.964496 915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93d290968bfd4b0e9bd37066b18d4517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:59.964726 642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93d290968bfd4b0e9bd37066b18d4517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:23:59.967941 1011 tablet_bootstrap.cc:492] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc: Bootstrap starting.
I20260619 08:23:59.968283 1010 tablet_bootstrap.cc:492] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f: Bootstrap starting.
I20260619 08:23:59.968297 1012 tablet_bootstrap.cc:492] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195: Bootstrap starting.
I20260619 08:23:59.969106 1012 tablet_bootstrap.cc:654] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:59.969218 1010 tablet_bootstrap.cc:654] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:59.969512 1010 log.cc:824] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:59.969535 1012 log.cc:824] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:59.970446 1011 tablet_bootstrap.cc:654] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc: Neither blocks nor log segments found. Creating new log.
I20260619 08:23:59.970465 1010 tablet_bootstrap.cc:492] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f: No bootstrap required, opened a new log
I20260619 08:23:59.970522 1010 ts_tablet_manager.cc:1397] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:23:59.970585 1012 tablet_bootstrap.cc:492] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195: No bootstrap required, opened a new log
I20260619 08:23:59.970659 1012 ts_tablet_manager.cc:1397] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:23:59.970911 1011 log.cc:824] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc: Log is configured to *not* fsync() on all Append() calls
I20260619 08:23:59.971778 1011 tablet_bootstrap.cc:492] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc: No bootstrap required, opened a new log
I20260619 08:23:59.971843 1011 ts_tablet_manager.cc:1397] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 08:23:59.972950 1010 raft_consensus.cc:348] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } }
I20260619 08:23:59.973073 1010 raft_consensus.cc:374] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:59.973094 1010 raft_consensus.cc:729] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d9893d7478843c3b96c5b842fc4d11f, State: Initialized, Role: FOLLOWER
I20260619 08:23:59.973219 1010 consensus_queue.cc:260] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } }
I20260619 08:23:59.973363 1012 raft_consensus.cc:348] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } }
I20260619 08:23:59.973500 1010 ts_tablet_manager.cc:1428] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:23:59.973510 1012 raft_consensus.cc:374] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:59.973546 1012 raft_consensus.cc:729] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e891c0b19a604149b7038f924fb83195, State: Initialized, Role: FOLLOWER
I20260619 08:23:59.973670 1012 consensus_queue.cc:260] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } }
I20260619 08:23:59.973711 982 heartbeater.cc:503] Master 127.25.127.190:35483 was elected leader, sending a full tablet report...
I20260619 08:23:59.973927 1012 ts_tablet_manager.cc:1428] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 08:23:59.974792 709 heartbeater.cc:503] Master 127.25.127.190:35483 was elected leader, sending a full tablet report...
I20260619 08:23:59.974959 1011 raft_consensus.cc:348] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } }
I20260619 08:23:59.975119 1011 raft_consensus.cc:374] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:23:59.975163 1011 raft_consensus.cc:729] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f897dd709d42f48d6b51b65720e9bc, State: Initialized, Role: FOLLOWER
I20260619 08:23:59.975443 1011 consensus_queue.cc:260] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } }
I20260619 08:23:59.975747 1011 ts_tablet_manager.cc:1428] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260619 08:23:59.975926 846 heartbeater.cc:503] Master 127.25.127.190:35483 was elected leader, sending a full tablet report...
W20260619 08:24:00.045414 848 tablet.cc:2369] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:24:00.058913 1016 raft_consensus.cc:482] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:24:00.059096 1016 raft_consensus.cc:504] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } }
I20260619 08:24:00.059532 1016 leader_election.cc:288] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e891c0b19a604149b7038f924fb83195 (127.25.127.129:41471), 15f897dd709d42f48d6b51b65720e9bc (127.25.127.130:34349)
I20260619 08:24:00.063855 1008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:00.060691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:44693 (local address 127.25.127.130:34349)
0619 08:24:00.060746 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:00.060748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:00.060757 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:24:00.060786 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:00.060787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:00.060796 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:00.060851 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:00.061063 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:00.061784 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:00.063109 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:00.063471 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:00.063565 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:00.063575 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:00.063741 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 08:24:00.063781 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 08:24:00.063858 1007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:00.059759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:58207 (local address 127.25.127.129:41471)
0619 08:24:00.059834 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:00.059836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:00.060444 (+ 608us) server_negotiation.cc:407] Connection header received
0619 08:24:00.060522 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:00.060523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:00.060534 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:00.060599 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:00.060890 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:00.061666 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:00.063049 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:00.063474 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:00.063568 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:00.063577 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:00.063743 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 08:24:00.063787 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 08:24:00.063866 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:00.059697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:41471 (local address 127.25.127.131:58207)
0619 08:24:00.060235 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:00.060248 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:00.060312 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:00.060747 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:00.060748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:00.060750 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:00.060825 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:00.060827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:00.061772 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:00.061774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:00.062934 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:00.062937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:00.063567 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:00.063568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:00.063650 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:00.063655 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:24:00.063735 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 08:24:00.063780 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516}
I20260619 08:24:00.063926 986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:00.060571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:34349 (local address 127.25.127.131:44693)
0619 08:24:00.060619 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:00.060630 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:00.060681 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:00.060906 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:00.060907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:00.060909 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:00.060987 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:00.060989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:00.061885 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:00.061886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:00.062807 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:00.062810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:00.063562 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:00.063563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:00.063642 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:00.063646 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:24:00.063731 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 08:24:00.063775 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":1}
I20260619 08:24:00.064343 662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93d290968bfd4b0e9bd37066b18d4517" candidate_uuid: "2d9893d7478843c3b96c5b842fc4d11f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e891c0b19a604149b7038f924fb83195" is_pre_election: true
I20260619 08:24:00.064457 799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93d290968bfd4b0e9bd37066b18d4517" candidate_uuid: "2d9893d7478843c3b96c5b842fc4d11f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f897dd709d42f48d6b51b65720e9bc" is_pre_election: true
I20260619 08:24:00.064548 662 raft_consensus.cc:2454] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d9893d7478843c3b96c5b842fc4d11f in term 0.
I20260619 08:24:00.064647 799 raft_consensus.cc:2454] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d9893d7478843c3b96c5b842fc4d11f in term 0.
I20260619 08:24:00.064783 870 leader_election.cc:302] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d9893d7478843c3b96c5b842fc4d11f, e891c0b19a604149b7038f924fb83195; no voters:
I20260619 08:24:00.064965 1016 raft_consensus.cc:2781] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:24:00.065032 1016 raft_consensus.cc:482] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:24:00.065081 1016 raft_consensus.cc:3037] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:00.065827 1016 raft_consensus.cc:504] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } }
I20260619 08:24:00.065982 1016 leader_election.cc:288] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [CANDIDATE]: Term 1 election: Requested vote from peers e891c0b19a604149b7038f924fb83195 (127.25.127.129:41471), 15f897dd709d42f48d6b51b65720e9bc (127.25.127.130:34349)
I20260619 08:24:00.066154 799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93d290968bfd4b0e9bd37066b18d4517" candidate_uuid: "2d9893d7478843c3b96c5b842fc4d11f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f897dd709d42f48d6b51b65720e9bc"
I20260619 08:24:00.066164 662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93d290968bfd4b0e9bd37066b18d4517" candidate_uuid: "2d9893d7478843c3b96c5b842fc4d11f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e891c0b19a604149b7038f924fb83195"
I20260619 08:24:00.066232 799 raft_consensus.cc:3037] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:00.066263 662 raft_consensus.cc:3037] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:00.066932 799 raft_consensus.cc:2454] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d9893d7478843c3b96c5b842fc4d11f in term 1.
I20260619 08:24:00.067029 662 raft_consensus.cc:2454] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d9893d7478843c3b96c5b842fc4d11f in term 1.
I20260619 08:24:00.067108 869 leader_election.cc:302] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f897dd709d42f48d6b51b65720e9bc, 2d9893d7478843c3b96c5b842fc4d11f; no voters:
I20260619 08:24:00.067224 1016 raft_consensus.cc:2781] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:24:00.067395 1016 raft_consensus.cc:686] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [term 1 LEADER]: Becoming Leader. State: Replica: 2d9893d7478843c3b96c5b842fc4d11f, State: Running, Role: LEADER
I20260619 08:24:00.067505 1016 consensus_queue.cc:237] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } }
I20260619 08:24:00.068635 32206 catalog_manager.cc:5495] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d9893d7478843c3b96c5b842fc4d11f (127.25.127.131). New cstate: current_term: 1 leader_uuid: "2d9893d7478843c3b96c5b842fc4d11f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } health_report { overall_health: UNKNOWN } } }
I20260619 08:24:00.078343 1009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:00.076130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43000 (local address 127.25.127.131:39721)
0619 08:24:00.076209 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:00.076211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:00.076222 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:24:00.076266 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:00.076267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:00.076282 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:00.076341 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:24:00.076464 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:00.076909 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:00.077616 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:00.077803 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:00.077858 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:00.078034 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:24:00.078128 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:24:00.078193 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:00.078230 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 08:24:00.078248 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 08:24:00.080103 799 raft_consensus.cc:1261] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc [term 1 FOLLOWER]: Refusing update from remote peer 2d9893d7478843c3b96c5b842fc4d11f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:24:00.080127 662 raft_consensus.cc:1261] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195 [term 1 FOLLOWER]: Refusing update from remote peer 2d9893d7478843c3b96c5b842fc4d11f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:24:00.080469 1016 consensus_queue.cc:1035] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [LEADER]: Connected to new peer: Peer: permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:24:00.080639 1019 consensus_queue.cc:1035] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:24:00.084237 1023 mvcc.cc:205] Tried to move back new op lower bound from 7298488074565492736 to 7298488074517291008. Current Snapshot: MvccSnapshot[applied={T|T < 7298488074565492736}]
I20260619 08:24:00.085131 1022 mvcc.cc:205] Tried to move back new op lower bound from 7298488074565492736 to 7298488074517291008. Current Snapshot: MvccSnapshot[applied={T|T < 7298488074565492736 or (T in {7298488074565492736})}]
I20260619 08:24:00.085935 32205 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:24:00.086050 32205 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:24:00.087507 32205 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a382c3e1cdce42d89abf62d01255925d: Sending DeleteTablet for 3 replicas of tablet 93d290968bfd4b0e9bd37066b18d4517
I20260619 08:24:00.087812 915 tablet_service.cc:1514] Processing DeleteTablet for tablet 93d290968bfd4b0e9bd37066b18d4517 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:00 UTC) from {username='oryx', principal='oryx/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:42990
I20260619 08:24:00.087819 642 tablet_service.cc:1514] Processing DeleteTablet for tablet 93d290968bfd4b0e9bd37066b18d4517 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:00 UTC) from {username='oryx', principal='oryx/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:38812
I20260619 08:24:00.088011 779 tablet_service.cc:1514] Processing DeleteTablet for tablet 93d290968bfd4b0e9bd37066b18d4517 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:00 UTC) from {username='oryx', principal='oryx/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:45560
I20260619 08:24:00.088131 1032 tablet_replica.cc:330] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195: stopping tablet replica
I20260619 08:24:00.088264 1032 raft_consensus.cc:2229] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:24:00.088372 1032 raft_consensus.cc:2258] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:24:00.088690 1033 tablet_replica.cc:330] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc: stopping tablet replica
I20260619 08:24:00.088804 1033 raft_consensus.cc:2229] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:24:00.088907 1033 raft_consensus.cc:2258] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:24:00.089100 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 32174
I20260619 08:24:00.089151 1032 ts_tablet_manager.cc:1905] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:00.091086 1032 ts_tablet_manager.cc:1918] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:24:00.091166 1032 log.cc:1196] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-0/wal/wals/93d290968bfd4b0e9bd37066b18d4517
I20260619 08:24:00.091451 1032 ts_tablet_manager.cc:1939] T 93d290968bfd4b0e9bd37066b18d4517 P e891c0b19a604149b7038f924fb83195: Deleting consensus metadata
I20260619 08:24:00.092584 1033 ts_tablet_manager.cc:1905] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:00.094403 1033 ts_tablet_manager.cc:1918] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:24:00.094491 1033 log.cc:1196] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/93d290968bfd4b0e9bd37066b18d4517
I20260619 08:24:00.094796 1033 ts_tablet_manager.cc:1939] T 93d290968bfd4b0e9bd37066b18d4517 P 15f897dd709d42f48d6b51b65720e9bc: Deleting consensus metadata
I20260619 08:24:00.095017 1031 tablet_replica.cc:330] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f: stopping tablet replica
I20260619 08:24:00.095121 1031 raft_consensus.cc:2229] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:24:00.095235 1031 raft_consensus.cc:2258] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 08:24:00.095909 735 connection.cc:535] server connection from 127.0.0.1:45560 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 08:24:00.096093 595 connection.cc:535] server connection from 127.0.0.1:38812 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 08:24:00.096256 1031 ts_tablet_manager.cc:1905] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:00.097967 1031 ts_tablet_manager.cc:1918] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:24:00.098057 1031 log.cc:1196] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/93d290968bfd4b0e9bd37066b18d4517
I20260619 08:24:00.098344 1031 ts_tablet_manager.cc:1939] T 93d290968bfd4b0e9bd37066b18d4517 P 2d9893d7478843c3b96c5b842fc4d11f: Deleting consensus metadata
W20260619 08:24:00.098675 870 connection.cc:441] server connection from 127.0.0.1:42990 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42990 (request call id 1) could send its response
I20260619 08:24:00.101379 992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:00.088261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39152 (local address 127.25.127.188:34461)
0619 08:24:00.088303 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:00.088305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:00.092380 (+ 4075us) server_negotiation.cc:407] Connection header received
0619 08:24:00.092482 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:00.092483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:00.092498 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:00.092526 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:00.093445 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:00.094308 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:00.095778 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:00.096095 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:00.096217 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:00.098151 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:00.098152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:00.098153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:00.098161 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:00.100042 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:00.100488 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:00.100490 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:00.100490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:00.100560 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:00.100773 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:00.100774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:00.100774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:00.100903 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:00.100980 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:00.101189 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 08:24:00.101279 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:24:00.105727 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:00.098296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39164 (local address 127.25.127.188:34461)
0619 08:24:00.098338 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:00.098340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:00.098348 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:24:00.098369 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:00.098370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:00.098378 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:00.098427 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:00.099131 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:00.099860 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:00.100993 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:00.101303 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:00.101341 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:00.103153 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:00.103155 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:00.103155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:00.103162 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:00.104514 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:00.104924 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:00.104925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:00.104926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:00.104984 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:00.105195 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:00.105197 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:00.105197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:00.105318 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:00.105426 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:00.105529 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 08:24:00.105604 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:24:00.106820 993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:00.098255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41254 (local address 127.25.127.189:46777)
0619 08:24:00.098344 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:00.098346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:00.098353 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:24:00.098376 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:00.098376 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:00.098388 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:00.098436 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:00.099112 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:00.099870 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:00.101289 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:00.101586 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:00.101640 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:00.103346 (+ 1706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:00.103347 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:00.103348 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:00.103615 (+ 267us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:00.105725 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:00.106100 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:00.106101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:00.106101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:00.106162 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:00.106329 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:00.106330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:00.106330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:00.106462 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:00.106570 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:00.106632 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 08:24:00.106723 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 08:24:01.084682 1041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:01.084348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.131:39533)
0619 08:24:01.084501 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:01.084620 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
W20260619 08:24:01.084959 982 heartbeater.cc:644] Failed to heartbeat to 127.25.127.190:35483 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
I20260619 08:24:01.085332 1042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:01.085015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.130:53503)
0619 08:24:01.085212 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:01.085289 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":90,"threads_started":1}
W20260619 08:24:01.085529 846 heartbeater.cc:644] Failed to heartbeat to 127.25.127.190:35483 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
I20260619 08:24:01.085685 1043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:01.085285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.129:35461)
0619 08:24:01.085563 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:01.085637 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":49,"threads_started":1}
W20260619 08:24:01.085850 709 heartbeater.cc:644] Failed to heartbeat to 127.25.127.190:35483 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
I20260619 08:24:01.588129 1044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a382c3e1cdce42d89abf62d01255925d)
I20260619 08:24:01.588337 1044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } }
I20260619 08:24:01.588502 1045 raft_consensus.cc:482] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a382c3e1cdce42d89abf62d01255925d)
I20260619 08:24:01.588599 1045 raft_consensus.cc:504] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } }
I20260619 08:24:01.588671 1044 leader_election.cc:288] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a382c3e1cdce42d89abf62d01255925d (127.25.127.190:35483), beb8df1f97bf484b822e808559f5a228 (127.25.127.188:34461)
I20260619 08:24:01.588816 1045 leader_election.cc:288] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a382c3e1cdce42d89abf62d01255925d (127.25.127.190:35483), 473f4c6d5426459a9c73ab26656abe8d (127.25.127.189:46777)
I20260619 08:24:01.589026 1046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:01.588761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:43924)
0619 08:24:01.588927 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:01.588979 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":68,"threads_started":1}
W20260619 08:24:01.589169 321 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
I20260619 08:24:01.589402 1048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:01.588979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:43938)
0619 08:24:01.589293 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:01.589356 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":76,"threads_started":1}
W20260619 08:24:01.589540 515 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
I20260619 08:24:01.589669 1047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:01.589538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:43954)
0619 08:24:01.589596 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:01.589640 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 08:24:01.589759 321 leader_election.cc:334] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a382c3e1cdce42d89abf62d01255925d (127.25.127.190:35483): Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
I20260619 08:24:01.590088 1048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:01.589979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:43970)
0619 08:24:01.590032 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:01.590062 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 08:24:01.590205 515 leader_election.cc:334] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a382c3e1cdce42d89abf62d01255925d (127.25.127.190:35483): Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
I20260619 08:24:01.592358 1049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:01.588977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:46777 (local address 127.0.0.1:41258)
0619 08:24:01.589160 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:01.589170 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:01.589225 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:01.589431 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:01.589433 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:01.589436 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:01.589659 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:01.589662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:01.590627 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:01.590628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:01.591487 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:01.591489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:01.592036 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:01.592037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:01.592106 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:01.592110 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:24:01.592227 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 08:24:01.592276 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":59,"threads_started":1}
I20260619 08:24:01.592365 1050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:01.589056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41258 (local address 127.25.127.189:46777)
0619 08:24:01.589283 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:01.589285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:01.589293 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:24:01.589318 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:01.589319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:01.589329 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:01.589796 (+ 467us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:01.589856 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:01.590544 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:01.591575 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:01.591954 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:01.592038 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:01.592047 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:01.592235 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 08:24:01.592278 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":38,"threads_started":1}
I20260619 08:24:01.592362 1051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:01.589073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39168 (local address 127.25.127.188:34461)
0619 08:24:01.589304 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:01.589306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:01.589313 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:24:01.589335 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:01.589335 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:01.589345 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:01.589411 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:01.589885 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:01.590538 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:01.591629 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:01.592005 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:01.592086 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:01.592093 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:01.592249 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 08:24:01.592289 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":33,"threads_started":1}
I20260619 08:24:01.592370 1046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:01.588817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:34461 (local address 127.0.0.1:39168)
0619 08:24:01.589097 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:01.589107 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:01.589195 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:01.589739 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:01.589739 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:01.589742 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:01.589795 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:01.589797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:01.590623 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:01.590624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:01.591539 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:01.591541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:01.592094 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:01.592095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:01.592181 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:01.592185 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:24:01.592243 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 08:24:01.592285 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":49,"threads_started":1}
I20260619 08:24:01.592581 345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "beb8df1f97bf484b822e808559f5a228" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "473f4c6d5426459a9c73ab26656abe8d" is_pre_election: true
I20260619 08:24:01.592689 345 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate beb8df1f97bf484b822e808559f5a228 in term 1.
I20260619 08:24:01.592859 514 leader_election.cc:302] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 473f4c6d5426459a9c73ab26656abe8d, beb8df1f97bf484b822e808559f5a228; no voters: a382c3e1cdce42d89abf62d01255925d
I20260619 08:24:01.592831 539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "473f4c6d5426459a9c73ab26656abe8d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "beb8df1f97bf484b822e808559f5a228" is_pre_election: true
I20260619 08:24:01.592942 539 raft_consensus.cc:2454] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 473f4c6d5426459a9c73ab26656abe8d in term 1.
I20260619 08:24:01.593040 1045 raft_consensus.cc:2781] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 08:24:01.593122 1045 raft_consensus.cc:482] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a382c3e1cdce42d89abf62d01255925d)
I20260619 08:24:01.593125 319 leader_election.cc:302] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 473f4c6d5426459a9c73ab26656abe8d, beb8df1f97bf484b822e808559f5a228; no voters: a382c3e1cdce42d89abf62d01255925d
I20260619 08:24:01.593155 1045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:24:01.593235 1044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 08:24:01.593276 1044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 FOLLOWER]: Starting leader election (detected failure of leader a382c3e1cdce42d89abf62d01255925d)
I20260619 08:24:01.593302 1044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:24:01.594043 1045 raft_consensus.cc:504] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } }
I20260619 08:24:01.594094 1044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } }
I20260619 08:24:01.594206 1044 leader_election.cc:288] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [CANDIDATE]: Term 2 election: Requested vote from peers a382c3e1cdce42d89abf62d01255925d (127.25.127.190:35483), beb8df1f97bf484b822e808559f5a228 (127.25.127.188:34461)
I20260619 08:24:01.594204 1045 leader_election.cc:288] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [CANDIDATE]: Term 2 election: Requested vote from peers a382c3e1cdce42d89abf62d01255925d (127.25.127.190:35483), 473f4c6d5426459a9c73ab26656abe8d (127.25.127.189:46777)
I20260619 08:24:01.594368 345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "beb8df1f97bf484b822e808559f5a228" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "473f4c6d5426459a9c73ab26656abe8d"
I20260619 08:24:01.594447 1049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:01.594326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:43980)
0619 08:24:01.594407 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:01.594427 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260619 08:24:01.594502 345 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate beb8df1f97bf484b822e808559f5a228 in current term 2: Already voted for candidate 473f4c6d5426459a9c73ab26656abe8d in this term.
I20260619 08:24:01.594437 1046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:01.594326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:43982)
0619 08:24:01.594363 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:01.594392 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 08:24:01.594592 539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "473f4c6d5426459a9c73ab26656abe8d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "beb8df1f97bf484b822e808559f5a228"
I20260619 08:24:01.594689 539 raft_consensus.cc:2379] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 473f4c6d5426459a9c73ab26656abe8d in current term 2: Already voted for candidate beb8df1f97bf484b822e808559f5a228 in this term.
I20260619 08:24:01.594749 1049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:01.594664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:43996)
0619 08:24:01.594703 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:01.594722 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 08:24:01.594822 1047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:01.594732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:44002)
0619 08:24:01.594772 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:01.594797 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 08:24:01.594879 321 leader_election.cc:334] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a382c3e1cdce42d89abf62d01255925d (127.25.127.190:35483): Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
W20260619 08:24:01.594902 515 leader_election.cc:334] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a382c3e1cdce42d89abf62d01255925d (127.25.127.190:35483): Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
I20260619 08:24:01.594944 515 leader_election.cc:302] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: beb8df1f97bf484b822e808559f5a228; no voters: 473f4c6d5426459a9c73ab26656abe8d, a382c3e1cdce42d89abf62d01255925d
I20260619 08:24:01.594935 319 leader_election.cc:302] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 473f4c6d5426459a9c73ab26656abe8d; no voters: a382c3e1cdce42d89abf62d01255925d, beb8df1f97bf484b822e808559f5a228
I20260619 08:24:01.595006 1044 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 08:24:01.595093 1045 raft_consensus.cc:2726] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 08:24:02.085729 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:02.085423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.131:35149)
0619 08:24:02.085620 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:02.085670 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":94,"threads_started":1}
I20260619 08:24:02.086045 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:02.085816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.130:42473)
0619 08:24:02.085937 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:02.085973 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":52,"threads_started":1}
I20260619 08:24:02.086484 1056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:02.086284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.129:45081)
0619 08:24:02.086396 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:02.086435 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":61,"threads_started":1}
I20260619 08:24:03.086444 1057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.086204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.131:42451)
0619 08:24:03.086334 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.086376 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260619 08:24:03.086737 1058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.086503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.130:51619)
0619 08:24:03.086650 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.086687 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260619 08:24:03.087185 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.086988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.129:44633)
0619 08:24:03.087122 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.087152 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":90,"threads_started":1}
I20260619 08:24:03.197702 1060 raft_consensus.cc:482] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:24:03.197800 1060 raft_consensus.cc:504] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } }
I20260619 08:24:03.198031 1060 leader_election.cc:288] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a382c3e1cdce42d89abf62d01255925d (127.25.127.190:35483), beb8df1f97bf484b822e808559f5a228 (127.25.127.188:34461)
I20260619 08:24:03.198321 539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "473f4c6d5426459a9c73ab26656abe8d" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "beb8df1f97bf484b822e808559f5a228" is_pre_election: true
I20260619 08:24:03.198439 539 raft_consensus.cc:2454] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 473f4c6d5426459a9c73ab26656abe8d in term 2.
I20260619 08:24:03.198441 1061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.198211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:44226)
0619 08:24:03.198339 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.198396 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":76,"threads_started":1}
I20260619 08:24:03.198599 319 leader_election.cc:302] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 473f4c6d5426459a9c73ab26656abe8d, beb8df1f97bf484b822e808559f5a228; no voters:
I20260619 08:24:03.198691 1060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 08:24:03.198730 1060 raft_consensus.cc:482] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:24:03.198755 1060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 2 FOLLOWER]: Advancing to term 3
I20260619 08:24:03.198925 1061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.198825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:44234)
0619 08:24:03.198861 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.198903 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260619 08:24:03.199033 321 leader_election.cc:334] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a382c3e1cdce42d89abf62d01255925d (127.25.127.190:35483): Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
I20260619 08:24:03.199646 1060 raft_consensus.cc:504] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } }
I20260619 08:24:03.199788 1060 leader_election.cc:288] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [CANDIDATE]: Term 3 election: Requested vote from peers a382c3e1cdce42d89abf62d01255925d (127.25.127.190:35483), beb8df1f97bf484b822e808559f5a228 (127.25.127.188:34461)
I20260619 08:24:03.199965 1061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.199864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:44238)
0619 08:24:03.199903 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.199938 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260619 08:24:03.199956 539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "473f4c6d5426459a9c73ab26656abe8d" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "beb8df1f97bf484b822e808559f5a228"
I20260619 08:24:03.200060 539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 2 FOLLOWER]: Advancing to term 3
I20260619 08:24:03.200304 1061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.200206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:44248)
0619 08:24:03.200250 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.200275 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 08:24:03.200373 321 leader_election.cc:334] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a382c3e1cdce42d89abf62d01255925d (127.25.127.190:35483): Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
I20260619 08:24:03.200780 539 raft_consensus.cc:2454] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 473f4c6d5426459a9c73ab26656abe8d in term 3.
I20260619 08:24:03.200973 319 leader_election.cc:302] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 473f4c6d5426459a9c73ab26656abe8d, beb8df1f97bf484b822e808559f5a228; no voters: a382c3e1cdce42d89abf62d01255925d
I20260619 08:24:03.201088 1060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 3 FOLLOWER]: Leader election won for term 3
I20260619 08:24:03.201205 1060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [term 3 LEADER]: Becoming Leader. State: Replica: 473f4c6d5426459a9c73ab26656abe8d, State: Running, Role: LEADER
I20260619 08:24:03.201319 1060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } }
I20260619 08:24:03.201707 1063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 473f4c6d5426459a9c73ab26656abe8d. Latest consensus state: current_term: 3 leader_uuid: "473f4c6d5426459a9c73ab26656abe8d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } } }
I20260619 08:24:03.201790 1063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [sys.catalog]: This master's current role is: LEADER
I20260619 08:24:03.201960 1065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:24:03.202370 1065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:24:03.202463 1065 catalog_manager.cc:1259] Loaded cluster ID: 5f7f4708a0c94aab8e7bf11e4f6f51f7
I20260619 08:24:03.202495 1065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:24:03.202674 1065 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:24:03.202769 1065 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: Loaded TSK: 0
I20260619 08:24:03.202910 1065 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:24:03.263427 335 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:24:03.265221 335 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:24:03.266322 1061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.266178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:44262)
0619 08:24:03.266246 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.266287 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 08:24:03.266326 539 raft_consensus.cc:1261] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [term 3 FOLLOWER]: Refusing update from remote peer 473f4c6d5426459a9c73ab26656abe8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260619 08:24:03.266537 1063 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260619 08:24:03.266714 1061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.266603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.0.0.1:44264)
0619 08:24:03.266657 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.266690 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 08:24:03.266855 321 consensus_peers.cc:487] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d -> Peer a382c3e1cdce42d89abf62d01255925d (127.25.127.190:35483): Couldn't send request to peer a382c3e1cdce42d89abf62d01255925d. Status: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 08:24:03.267261 1067 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 473f4c6d5426459a9c73ab26656abe8d. Latest consensus state: current_term: 3 leader_uuid: "473f4c6d5426459a9c73ab26656abe8d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } } }
I20260619 08:24:03.267374 1067 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:24:03.267802 1060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "473f4c6d5426459a9c73ab26656abe8d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } } }
I20260619 08:24:03.267892 1060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d [sys.catalog]: This master's current role is: LEADER
I20260619 08:24:03.267976 1067 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "473f4c6d5426459a9c73ab26656abe8d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a382c3e1cdce42d89abf62d01255925d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 35483 } } peers { permanent_uuid: "473f4c6d5426459a9c73ab26656abe8d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 46777 } attrs { promote: false } } peers { permanent_uuid: "beb8df1f97bf484b822e808559f5a228" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 34461 } attrs { promote: false } } }
I20260619 08:24:03.268066 1067 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb8df1f97bf484b822e808559f5a228 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:24:03.272991 1061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.269629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:41471 (local address 127.0.0.1:38816)
0619 08:24:03.269689 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.269701 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:03.269752 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:03.270167 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:03.270168 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:03.270169 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:03.270225 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:03.270227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.271081 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.271082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:03.271992 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:03.271995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.272776 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.272777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:03.272845 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:03.272850 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:24:03.272878 (+ 28us) client_negotiation.cc:240] Negotiation successful
0619 08:24:03.272920 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 08:24:03.273361 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.269705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:34349 (local address 127.0.0.1:45568)
0619 08:24:03.270139 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.270150 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:03.270191 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:03.270395 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:03.270395 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:03.270398 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:03.270469 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:03.270470 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.271365 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.271366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:03.272548 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:03.272550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.273088 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.273089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:03.273150 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:03.273154 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:24:03.273230 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 08:24:03.273271 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":77,"threads_started":1}
I20260619 08:24:03.273505 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.269703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:39721 (local address 127.0.0.1:43006)
0619 08:24:03.269844 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.269854 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:03.269906 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:03.270118 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:03.270119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:03.270122 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:03.270328 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:03.270330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.271278 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.271279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:03.272461 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:03.272464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.273227 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.273228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:03.273279 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:03.273282 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:24:03.273336 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:24:03.273367 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":2,"thread_start_us":88,"threads_started":1}
I20260619 08:24:03.273522 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.270029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45568 (local address 127.25.127.130:34349)
0619 08:24:03.270225 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:03.270227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:03.270235 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:24:03.270267 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:03.270268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:03.270278 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:03.270347 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:03.270547 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.271291 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.272613 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.272996 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.273080 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:03.273088 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:03.273401 (+ 313us) server_negotiation.cc:299] Negotiation successful
0619 08:24:03.273450 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":77,"threads_started":1}
I20260619 08:24:03.273556 1074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.269760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43006 (local address 127.25.127.131:39721)
0619 08:24:03.269867 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:03.269870 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:03.269893 (+ 23us) server_negotiation.cc:407] Connection header received
0619 08:24:03.269986 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:03.269987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:03.269999 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:03.270064 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:03.270409 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.271194 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.272545 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.273148 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.273222 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:03.273231 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:03.273348 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 08:24:03.273396 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":51,"threads_started":1}
I20260619 08:24:03.273728 779 tablet_service.cc:1467] Processing CreateTablet for tablet a33417cb64fe4f8ab2332a6e0fe221b4 (DEFAULT_TABLE table=test-table [id=86616b97a8394a5f97cd8938085b9136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:24:03.273867 779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a33417cb64fe4f8ab2332a6e0fe221b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:03.273919 915 tablet_service.cc:1467] Processing CreateTablet for tablet a33417cb64fe4f8ab2332a6e0fe221b4 (DEFAULT_TABLE table=test-table [id=86616b97a8394a5f97cd8938085b9136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:24:03.273947 1075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.269844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38816 (local address 127.25.127.129:41471)
0619 08:24:03.269971 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:03.269973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:03.269983 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:24:03.270012 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:03.270013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:03.270024 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:03.270293 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:03.270334 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.270973 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.272077 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.272680 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.273357 (+ 677us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:03.273368 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:03.273385 (+ 17us) server_negotiation.cc:299] Negotiation successful
0619 08:24:03.273848 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
I20260619 08:24:03.274027 915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a33417cb64fe4f8ab2332a6e0fe221b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:03.274175 642 tablet_service.cc:1467] Processing CreateTablet for tablet a33417cb64fe4f8ab2332a6e0fe221b4 (DEFAULT_TABLE table=test-table [id=86616b97a8394a5f97cd8938085b9136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:24:03.274298 642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a33417cb64fe4f8ab2332a6e0fe221b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:03.275545 1077 tablet_bootstrap.cc:492] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc: Bootstrap starting.
I20260619 08:24:03.275672 1079 tablet_bootstrap.cc:492] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195: Bootstrap starting.
I20260619 08:24:03.275954 1077 tablet_bootstrap.cc:654] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:03.275991 1079 tablet_bootstrap.cc:654] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:03.276247 1078 tablet_bootstrap.cc:492] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f: Bootstrap starting.
I20260619 08:24:03.276574 1079 tablet_bootstrap.cc:492] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195: No bootstrap required, opened a new log
I20260619 08:24:03.276680 1078 tablet_bootstrap.cc:654] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:03.276679 1079 ts_tablet_manager.cc:1397] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:03.276783 1077 tablet_bootstrap.cc:492] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc: No bootstrap required, opened a new log
I20260619 08:24:03.276832 1077 ts_tablet_manager.cc:1397] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 08:24:03.276839 1079 raft_consensus.cc:348] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } }
I20260619 08:24:03.276921 1079 raft_consensus.cc:374] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:03.276942 1079 raft_consensus.cc:729] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e891c0b19a604149b7038f924fb83195, State: Initialized, Role: FOLLOWER
I20260619 08:24:03.276942 1077 raft_consensus.cc:348] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } }
I20260619 08:24:03.277012 1077 raft_consensus.cc:374] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:03.277043 1077 raft_consensus.cc:729] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f897dd709d42f48d6b51b65720e9bc, State: Initialized, Role: FOLLOWER
I20260619 08:24:03.277017 1079 consensus_queue.cc:260] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } }
I20260619 08:24:03.277124 1077 consensus_queue.cc:260] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } }
I20260619 08:24:03.277220 1079 ts_tablet_manager.cc:1428] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:03.277537 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.277432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.129:41069)
0619 08:24:03.277473 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.277510 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 08:24:03.277865 710 heartbeater.cc:503] Master 127.25.127.189:46777 was elected leader, sending a full tablet report...
I20260619 08:24:03.277930 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.277795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.129:60551)
0619 08:24:03.277832 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.277856 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 08:24:03.277969 1077 ts_tablet_manager.cc:1428] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 08:24:03.278340 1058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.278244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.130:60373)
0619 08:24:03.278285 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.278309 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 08:24:03.278460 847 heartbeater.cc:503] Master 127.25.127.189:46777 was elected leader, sending a full tablet report...
I20260619 08:24:03.278785 1058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.278671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.130:43851)
0619 08:24:03.278725 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.278756 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 08:24:03.279125 1078 tablet_bootstrap.cc:492] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f: No bootstrap required, opened a new log
I20260619 08:24:03.279170 1078 ts_tablet_manager.cc:1397] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 08:24:03.279290 1078 raft_consensus.cc:348] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } }
I20260619 08:24:03.279356 1078 raft_consensus.cc:374] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:03.279371 1078 raft_consensus.cc:729] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d9893d7478843c3b96c5b842fc4d11f, State: Initialized, Role: FOLLOWER
I20260619 08:24:03.279440 1078 consensus_queue.cc:260] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } }
I20260619 08:24:03.279635 1078 ts_tablet_manager.cc:1428] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:24:03.279933 1057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.279809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.131:54313)
0619 08:24:03.279872 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.279899 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 08:24:03.326601 1083 raft_consensus.cc:482] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:24:03.326741 1083 raft_consensus.cc:504] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } }
I20260619 08:24:03.327157 1083 leader_election.cc:288] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d9893d7478843c3b96c5b842fc4d11f (127.25.127.131:39721), 15f897dd709d42f48d6b51b65720e9bc (127.25.127.130:34349)
I20260619 08:24:03.330392 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.327415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:56853 (local address 127.25.127.130:34349)
0619 08:24:03.327458 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:03.327460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:03.327468 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:24:03.327568 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:03.327569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:03.327577 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:03.327652 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:03.327839 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.328471 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.329481 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.330025 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.330082 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:03.330090 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:03.330265 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 08:24:03.330313 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:24:03.330405 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.327314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:34349 (local address 127.25.127.129:56853)
0619 08:24:03.327382 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.327396 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:03.327464 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:03.327665 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:03.327666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:03.327668 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:03.327758 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:03.327759 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.328551 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.328552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:03.329401 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:03.329404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.330103 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.330105 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:03.330198 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:03.330202 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:24:03.330255 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 08:24:03.330307 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 08:24:03.330490 1074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.327408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:52501 (local address 127.25.127.131:39721)
0619 08:24:03.327453 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:03.327455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:03.327759 (+ 304us) server_negotiation.cc:407] Connection header received
0619 08:24:03.327792 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:03.327794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:03.327805 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:03.327879 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:03.328018 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.328578 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.329658 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.330096 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.330167 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:03.330179 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:03.330370 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 08:24:03.330415 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:24:03.330713 1086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.327315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:39721 (local address 127.25.127.129:52501)
0619 08:24:03.327674 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.327686 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:03.327748 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:03.327878 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:03.327878 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:03.327880 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:03.327919 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:03.327921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.328669 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.328670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:03.329572 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:03.329575 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.330205 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.330206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:03.330286 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:03.330291 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:24:03.330347 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 08:24:03.330386 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":150,"thread_start_us":133,"threads_started":1}
I20260619 08:24:03.330765 799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a33417cb64fe4f8ab2332a6e0fe221b4" candidate_uuid: "e891c0b19a604149b7038f924fb83195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f897dd709d42f48d6b51b65720e9bc" is_pre_election: true
I20260619 08:24:03.330885 799 raft_consensus.cc:2454] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e891c0b19a604149b7038f924fb83195 in term 0.
I20260619 08:24:03.331019 935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a33417cb64fe4f8ab2332a6e0fe221b4" candidate_uuid: "e891c0b19a604149b7038f924fb83195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d9893d7478843c3b96c5b842fc4d11f" is_pre_election: true
I20260619 08:24:03.331058 596 leader_election.cc:302] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f897dd709d42f48d6b51b65720e9bc, e891c0b19a604149b7038f924fb83195; no voters:
I20260619 08:24:03.331154 935 raft_consensus.cc:2454] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e891c0b19a604149b7038f924fb83195 in term 0.
I20260619 08:24:03.331223 1083 raft_consensus.cc:2781] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:24:03.331305 1083 raft_consensus.cc:482] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:24:03.331341 1083 raft_consensus.cc:3037] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:03.332038 1083 raft_consensus.cc:504] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } }
I20260619 08:24:03.332216 1083 leader_election.cc:288] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [CANDIDATE]: Term 1 election: Requested vote from peers 2d9893d7478843c3b96c5b842fc4d11f (127.25.127.131:39721), 15f897dd709d42f48d6b51b65720e9bc (127.25.127.130:34349)
I20260619 08:24:03.332358 935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a33417cb64fe4f8ab2332a6e0fe221b4" candidate_uuid: "e891c0b19a604149b7038f924fb83195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d9893d7478843c3b96c5b842fc4d11f"
I20260619 08:24:03.332362 799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a33417cb64fe4f8ab2332a6e0fe221b4" candidate_uuid: "e891c0b19a604149b7038f924fb83195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f897dd709d42f48d6b51b65720e9bc"
I20260619 08:24:03.332434 799 raft_consensus.cc:3037] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:03.332454 935 raft_consensus.cc:3037] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:03.333089 799 raft_consensus.cc:2454] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e891c0b19a604149b7038f924fb83195 in term 1.
I20260619 08:24:03.333195 935 raft_consensus.cc:2454] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e891c0b19a604149b7038f924fb83195 in term 1.
I20260619 08:24:03.333258 596 leader_election.cc:302] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f897dd709d42f48d6b51b65720e9bc, e891c0b19a604149b7038f924fb83195; no voters:
I20260619 08:24:03.333353 1083 raft_consensus.cc:2781] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:24:03.333496 1083 raft_consensus.cc:686] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [term 1 LEADER]: Becoming Leader. State: Replica: e891c0b19a604149b7038f924fb83195, State: Running, Role: LEADER
I20260619 08:24:03.333617 1083 consensus_queue.cc:237] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } }
I20260619 08:24:03.333839 1086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.333737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.129:50261)
0619 08:24:03.333771 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.333809 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 08:24:03.334467 335 catalog_manager.cc:5495] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 reported cstate change: term changed from 0 to 1, leader changed from <none> to e891c0b19a604149b7038f924fb83195 (127.25.127.129). New cstate: current_term: 1 leader_uuid: "e891c0b19a604149b7038f924fb83195" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e891c0b19a604149b7038f924fb83195" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 41471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 } health_report { overall_health: UNKNOWN } } }
I20260619 08:24:03.347443 1075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.344944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38828 (local address 127.25.127.129:41471)
0619 08:24:03.345009 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:03.345011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:03.345018 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:24:03.345054 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:03.345054 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:03.345073 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:03.345160 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:24:03.345324 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.345816 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.346575 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:03.346794 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:03.346882 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:03.347074 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:24:03.347184 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:24:03.347266 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:03.347320 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 08:24:03.347345 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:24:03.348867 799 raft_consensus.cc:1261] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc [term 1 FOLLOWER]: Refusing update from remote peer e891c0b19a604149b7038f924fb83195: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:24:03.348946 935 raft_consensus.cc:1261] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f [term 1 FOLLOWER]: Refusing update from remote peer e891c0b19a604149b7038f924fb83195: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:24:03.349164 1083 consensus_queue.cc:1035] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15f897dd709d42f48d6b51b65720e9bc" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:24:03.349320 1087 consensus_queue.cc:1035] T a33417cb64fe4f8ab2332a6e0fe221b4 P e891c0b19a604149b7038f924fb83195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d9893d7478843c3b96c5b842fc4d11f" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:24:03.350046 983 heartbeater.cc:503] Master 127.25.127.189:46777 was elected leader, sending a full tablet report...
I20260619 08:24:03.350119 1058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.350014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.130:45473)
0619 08:24:03.350048 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.350085 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260619 08:24:03.350414 1057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.350296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.131:42745)
0619 08:24:03.350347 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.350380 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 08:24:03.351773 1057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.351631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.131:38063)
0619 08:24:03.351715 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.351744 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260619 08:24:03.351944 1058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.351775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.130:55877)
0619 08:24:03.351886 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.351913 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260619 08:24:03.351954 1086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.351731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:35483 (local address 127.25.127.129:36153)
0619 08:24:03.351874 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.351914 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 08:24:03.352578 1091 mvcc.cc:205] Tried to move back new op lower bound from 7298488087954116608 to 7298488087895375872. Current Snapshot: MvccSnapshot[applied={T|T < 7298488087954116608}]
I20260619 08:24:03.354777 335 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:24:03.354882 335 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:24:03.356374 335 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 473f4c6d5426459a9c73ab26656abe8d: Sending DeleteTablet for 3 replicas of tablet a33417cb64fe4f8ab2332a6e0fe221b4
I20260619 08:24:03.356662 642 tablet_service.cc:1514] Processing DeleteTablet for tablet a33417cb64fe4f8ab2332a6e0fe221b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:03 UTC) from {username='oryx', principal='oryx/127.25.127.189@KRBTEST.COM'} at 127.0.0.1:38816
I20260619 08:24:03.356707 779 tablet_service.cc:1514] Processing DeleteTablet for tablet a33417cb64fe4f8ab2332a6e0fe221b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:03 UTC) from {username='oryx', principal='oryx/127.25.127.189@KRBTEST.COM'} at 127.0.0.1:45568
I20260619 08:24:03.356877 915 tablet_service.cc:1514] Processing DeleteTablet for tablet a33417cb64fe4f8ab2332a6e0fe221b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:03 UTC) from {username='oryx', principal='oryx/127.25.127.189@KRBTEST.COM'} at 127.0.0.1:43006
I20260619 08:24:03.357067 1102 tablet_replica.cc:330] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f: stopping tablet replica
I20260619 08:24:03.357175 1102 raft_consensus.cc:2229] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:24:03.357175 1101 tablet_replica.cc:330] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc: stopping tablet replica
I20260619 08:24:03.357232 1101 raft_consensus.cc:2229] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:24:03.357292 1102 raft_consensus.cc:2258] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:24:03.357410 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 484
I20260619 08:24:03.358042 1101 raft_consensus.cc:2258] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:24:03.358385 1101 ts_tablet_manager.cc:1905] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:03.359248 1102 ts_tablet_manager.cc:1905] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:03.360001 1101 ts_tablet_manager.cc:1918] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:24:03.360080 1101 log.cc:1196] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/a33417cb64fe4f8ab2332a6e0fe221b4
I20260619 08:24:03.360394 1101 ts_tablet_manager.cc:1939] T a33417cb64fe4f8ab2332a6e0fe221b4 P 15f897dd709d42f48d6b51b65720e9bc: Deleting consensus metadata
I20260619 08:24:03.360739 320 catalog_manager.cc:4841] TS 15f897dd709d42f48d6b51b65720e9bc (127.25.127.130:34349): tablet a33417cb64fe4f8ab2332a6e0fe221b4 (table test-table [id=86616b97a8394a5f97cd8938085b9136]) successfully deleted
I20260619 08:24:03.361047 1102 ts_tablet_manager.cc:1918] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:24:03.361114 1102 log.cc:1196] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/a33417cb64fe4f8ab2332a6e0fe221b4
I20260619 08:24:03.361346 1102 ts_tablet_manager.cc:1939] T a33417cb64fe4f8ab2332a6e0fe221b4 P 2d9893d7478843c3b96c5b842fc4d11f: Deleting consensus metadata
I20260619 08:24:03.361653 322 catalog_manager.cc:4841] TS 2d9893d7478843c3b96c5b842fc4d11f (127.25.127.131:39721): tablet a33417cb64fe4f8ab2332a6e0fe221b4 (table test-table [id=86616b97a8394a5f97cd8938085b9136]) successfully deleted
I20260619 08:24:03.363916 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 716
I20260619 08:24:03.363992 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:03.363794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:41471 (local address 127.0.0.1:38840)
0619 08:24:03.363867 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:03.363937 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.129:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 08:24:03.364571 321 catalog_manager.cc:4584] TS e891c0b19a604149b7038f924fb83195 (127.25.127.129:41471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a33417cb64fe4f8ab2332a6e0fe221b4: Network error: Client connection negotiation failed: client connection to 127.25.127.129:41471: connect: Connection refused (error 111)
I20260619 08:24:03.371501 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 852
I20260619 08:24:03.377372 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 32670
I20260619 08:24:03.382592 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 389
2026-06-19T08:24:03Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6242 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:24:03 dist-test-slave-5rhg krb5kdc[1105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:24:03 dist-test-slave-5rhg krb5kdc[1105](info): set up 2 sockets
Jun 19 08:24:03 dist-test-slave-5rhg krb5kdc[1105](info): commencing operation
krb5kdc: starting...
W20260619 08:24:04.441851 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:24:04 dist-test-slave-5rhg krb5kdc[1105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:24:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:24:04Z Disabled control of system clock
WARNING: no policy specified for oryx/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.190.keytab.
Entry for principal oryx/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.190.keytab.
I20260619 08:24:04.563732 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:38215
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.190.keytab
--principal=oryx/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:38215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:24:04.648443 1118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:04.648705 1118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:04.648746 1118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:04.650022 1118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:24:04.650096 1118 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 08:24:04.650120 1118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:04.650153 1118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:04.650202 1118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:24:04.650233 1118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:24:04.651863 1118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:38215
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.190.keytab
--principal=oryx/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:38215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:04.652228 1118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:04.652490 1118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:04.657433 1124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:04.657616 1118 server_base.cc:1034] running on GCE node
W20260619 08:24:04.657476 1126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:04.657653 1123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:04.658039 1118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:04.658361 1118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:04.659572 1118 hybrid_clock.cc:648] HybridClock initialized: now 1781857444659562 us; error 36 us; skew 500 ppm
Jun 19 08:24:04 dist-test-slave-5rhg krb5kdc[1105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857444, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:24:04.662366 1118 init.cc:377] Logged in from keytab as oryx/127.25.127.190@KRBTEST.COM (short username oryx)
I20260619 08:24:04.662899 1118 webserver.cc:460] Webserver started at http://127.25.127.190:42625/ using document root <none> and password file <none>
I20260619 08:24:04.663158 1118 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:04.663205 1118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:04.663379 1118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:04.664556 1118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "5695ede7410c499e9173818bbae01932"
format_stamp: "Formatted at 2026-06-19 08:24:04 on dist-test-slave-5rhg"
server_key: "5076a9edc98e9f2331f72b1c6982cfcc"
server_key_iv: "c75284bc3bb7337d16a74dea1700ee33"
server_key_version: "encryptionkey@0"
I20260619 08:24:04.664935 1118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "5695ede7410c499e9173818bbae01932"
format_stamp: "Formatted at 2026-06-19 08:24:04 on dist-test-slave-5rhg"
server_key: "5076a9edc98e9f2331f72b1c6982cfcc"
server_key_iv: "c75284bc3bb7337d16a74dea1700ee33"
server_key_version: "encryptionkey@0"
I20260619 08:24:04.666304 1118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 08:24:04.667117 1133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:04.667330 1118 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:04.667430 1118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "5695ede7410c499e9173818bbae01932"
format_stamp: "Formatted at 2026-06-19 08:24:04 on dist-test-slave-5rhg"
server_key: "5076a9edc98e9f2331f72b1c6982cfcc"
server_key_iv: "c75284bc3bb7337d16a74dea1700ee33"
server_key_version: "encryptionkey@0"
I20260619 08:24:04.667496 1118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:04.681885 1118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:04.683934 1118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:04.684057 1118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:04.688656 1118 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:38215
I20260619 08:24:04.688747 1195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:38215 every 8 connection(s)
I20260619 08:24:04.689073 1118 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:24:04.689771 1196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:04.692637 1196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932: Bootstrap starting.
I20260619 08:24:04.693367 1196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:04.693696 1196 log.cc:824] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932: Log is configured to *not* fsync() on all Append() calls
I20260619 08:24:04.694471 1196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932: No bootstrap required, opened a new log
I20260619 08:24:04.696775 1196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695ede7410c499e9173818bbae01932" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38215 } }
I20260619 08:24:04.696923 1196 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:04.696946 1196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5695ede7410c499e9173818bbae01932, State: Initialized, Role: FOLLOWER
I20260619 08:24:04.697132 1196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695ede7410c499e9173818bbae01932" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38215 } }
I20260619 08:24:04.697203 1196 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:24:04.697232 1196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:24:04.697278 1196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:04.697820 1196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695ede7410c499e9173818bbae01932" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38215 } }
I20260619 08:24:04.697938 1196 leader_election.cc:302] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5695ede7410c499e9173818bbae01932; no voters:
I20260619 08:24:04.698149 1196 leader_election.cc:288] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:24:04.698218 1199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:24:04.698396 1199 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932 [term 1 LEADER]: Becoming Leader. State: Replica: 5695ede7410c499e9173818bbae01932, State: Running, Role: LEADER
I20260619 08:24:04.698450 1196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:24:04.698495 1199 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695ede7410c499e9173818bbae01932" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38215 } }
I20260619 08:24:04.698900 1201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5695ede7410c499e9173818bbae01932. Latest consensus state: current_term: 1 leader_uuid: "5695ede7410c499e9173818bbae01932" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695ede7410c499e9173818bbae01932" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38215 } } }
I20260619 08:24:04.698997 1201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932 [sys.catalog]: This master's current role is: LEADER
I20260619 08:24:04.699087 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 1118
I20260619 08:24:04.699191 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:24:04.699088 1200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5695ede7410c499e9173818bbae01932" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695ede7410c499e9173818bbae01932" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38215 } } }
I20260619 08:24:04.699417 26110 external_mini_cluster.cc:1428] Setting key 7a5c83c7e3a4b5091bdd013643a8e5e6
I20260619 08:24:04.699445 1204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:24:04.699513 1200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932 [sys.catalog]: This master's current role is: LEADER
I20260619 08:24:04.700047 1204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:24:04.704609 1204 catalog_manager.cc:1347] Generated new cluster ID: 3012775fda394c5d80d927bade8ab416
I20260619 08:24:04.704695 1204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:24:04 dist-test-slave-5rhg krb5kdc[1105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM
I20260619 08:24:04.710012 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:04.700113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49228 (local address 127.25.127.190:38215)
0619 08:24:04.700577 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:04.700677 (+ 100us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:04.700711 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:24:04.700885 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:04.700893 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:04.700952 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:04.701118 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:04.701811 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:04.702722 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:04.703510 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:04.703985 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:04.704141 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:04.706652 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:04.706656 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:04.706657 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:04.706675 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:04.708436 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:04.708836 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:04.708839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:04.708840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:04.708935 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:04.709146 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:04.709147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:04.709148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:04.709408 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:04.709594 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:04.709661 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 08:24:04.709808 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":107,"threads_started":1}
I20260619 08:24:04.713989 1204 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:24:04.714694 1204 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:24:04.720459 1204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932: Generated new TSK 0
I20260619 08:24:04.720649 1204 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.129.keytab.
Entry for principal oryx/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.129.keytab.
I20260619 08:24:04.751133 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:38215
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.129.keytab
--principal=oryx/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:24:04.835322 1222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:04.835572 1222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:04.835611 1222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:04.836874 1222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:04.836951 1222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:04.836985 1222 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:24:04.837034 1222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:24:04.838618 1222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.129.keytab
--principal=oryx/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.127.190:38215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:04.838907 1222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:04.839156 1222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:04.844430 1228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:04.844427 1230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:04.844422 1227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:04.844746 1222 server_base.cc:1034] running on GCE node
I20260619 08:24:04.844995 1222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:04.845206 1222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:04.846350 1222 hybrid_clock.cc:648] HybridClock initialized: now 1781857444846331 us; error 34 us; skew 500 ppm
Jun 19 08:24:04 dist-test-slave-5rhg krb5kdc[1105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857444, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:24:04.848922 1222 init.cc:377] Logged in from keytab as oryx/127.25.127.129@KRBTEST.COM (short username oryx)
I20260619 08:24:04.849364 1222 webserver.cc:460] Webserver started at http://127.25.127.129:43661/ using document root <none> and password file <none>
I20260619 08:24:04.849624 1222 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:04.849671 1222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:04.849838 1222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:04.850922 1222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "a39e357cd3814cd69a9a5b91fb41b662"
format_stamp: "Formatted at 2026-06-19 08:24:04 on dist-test-slave-5rhg"
server_key: "17fee97f7943d68d1378ada5eb64f4fa"
server_key_iv: "9e2577569c48d18393fa140bb328ed0d"
server_key_version: "encryptionkey@0"
I20260619 08:24:04.851286 1222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "a39e357cd3814cd69a9a5b91fb41b662"
format_stamp: "Formatted at 2026-06-19 08:24:04 on dist-test-slave-5rhg"
server_key: "17fee97f7943d68d1378ada5eb64f4fa"
server_key_iv: "9e2577569c48d18393fa140bb328ed0d"
server_key_version: "encryptionkey@0"
I20260619 08:24:04.852764 1222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:24:04.853569 1237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:04.853753 1222 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:04.853861 1222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "a39e357cd3814cd69a9a5b91fb41b662"
format_stamp: "Formatted at 2026-06-19 08:24:04 on dist-test-slave-5rhg"
server_key: "17fee97f7943d68d1378ada5eb64f4fa"
server_key_iv: "9e2577569c48d18393fa140bb328ed0d"
server_key_version: "encryptionkey@0"
I20260619 08:24:04.853940 1222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:04.864552 1222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:04.866784 1222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:04.866955 1222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:04.867473 1222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:24:04.867539 1222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:04.867581 1222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:24:04.867630 1222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:04.874672 1222 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:43789
I20260619 08:24:04.874748 1352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:43789 every 8 connection(s)
Jun 19 08:24:04 dist-test-slave-5rhg krb5kdc[1105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857444, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM
I20260619 08:24:04.875109 1222 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:24:04.876475 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 1222
I20260619 08:24:04.876621 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:24:04.876835 26110 external_mini_cluster.cc:1428] Setting key 3dd4c3555369fca73952878fc14eded0
I20260619 08:24:04.878731 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:04.868227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:45699 (local address 127.25.127.190:38215)
0619 08:24:04.868290 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:04.868293 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:04.868508 (+ 215us) server_negotiation.cc:407] Connection header received
0619 08:24:04.869190 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:04.869191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:04.869209 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:04.869263 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:04.870147 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:04.871036 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:04.872302 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:04.872697 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:04.872785 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:04.875601 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:04.875603 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:04.875603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:04.875612 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:04.877314 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:04.877773 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:04.877774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:04.877774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:04.877826 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:04.878119 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:04.878119 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:04.878119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:04.878216 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:04.878265 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:04.878592 (+ 327us) server_negotiation.cc:299] Negotiation successful
0619 08:24:04.878665 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:24:04.879053 1246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:04.868210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:38215 (local address 127.25.127.129:45699)
0619 08:24:04.868386 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:04.868423 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:04.869023 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:04.869336 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:04.869341 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:04.869725 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:04.870018 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:04.870025 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:04.871143 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:04.871143 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:04.872208 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:04.872212 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:04.872798 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:04.872799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:04.872948 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:04.873551 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:04.873563 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:04.875514 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:04.877380 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:04.877388 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:04.877390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:04.877698 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:04.877890 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:04.877891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:04.877892 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:04.878030 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:04.878277 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:04.878283 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:04.878456 (+ 173us) client_negotiation.cc:769] Sending connection context
0619 08:24:04.878572 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 08:24:04.878757 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":84,"threads_started":1}
I20260619 08:24:04.879629 1353 heartbeater.cc:347] Connected to a master server at 127.25.127.190:38215
I20260619 08:24:04.879777 1353 heartbeater.cc:464] Registering TS with master...
I20260619 08:24:04.880033 1353 heartbeater.cc:511] Master 127.25.127.190:38215 requested a full tablet report, sending...
I20260619 08:24:04.880685 1150 ts_manager.cc:194] Registered new tserver with Master: a39e357cd3814cd69a9a5b91fb41b662 (127.25.127.129:43789)
I20260619 08:24:04.881389 1150 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:45699
WARNING: no policy specified for oryx/127.25.127.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.127.130@KRBTEST.COM" created.
I20260619 08:24:04.890290 1360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:04.880741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:38215 (local address 127.25.127.129:34571)
0619 08:24:04.884272 (+ 3531us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:04.884283 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:04.884376 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:04.884507 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:04.884508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:04.884710 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:04.884764 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:04.884766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:04.885525 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:04.885526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:04.886149 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:04.886152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:04.886522 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:04.886523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:04.886586 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:04.887022 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:04.887052 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:04.887412 (+ 360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:04.889225 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:04.889226 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:04.889226 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:04.889402 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:04.889617 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:04.889618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:04.889618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:04.889676 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:04.889933 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:04.889934 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:04.889985 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 08:24:04.890074 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 08:24:04.890182 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3496,"thread_start_us":69,"threads_started":1}
I20260619 08:24:04.890285 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:04.880896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:34571 (local address 127.25.127.190:38215)
0619 08:24:04.880933 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:04.880935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:04.884360 (+ 3425us) server_negotiation.cc:407] Connection header received
0619 08:24:04.884435 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:04.884436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:04.884447 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:04.884502 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:04.884843 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:04.885363 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:04.886227 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:04.886435 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:04.886489 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:04.887508 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:04.887510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:04.887510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:04.887516 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:04.889119 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:04.889487 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:04.889488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:04.889488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:04.889550 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:04.889747 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:04.889748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:04.889749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:04.889865 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:04.889942 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:04.890099 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 08:24:04.890182 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.130.keytab.
Entry for principal oryx/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.130.keytab.
I20260619 08:24:04.903280 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:38215
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.130.keytab
--principal=oryx/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:24:04.985433 1362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:04.985661 1362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:04.985687 1362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:04.986920 1362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:04.986965 1362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:04.986986 1362 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:24:04.987003 1362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:24:04.988667 1362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.130.keytab
--principal=oryx/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.127.190:38215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:04.988924 1362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:04.989171 1362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:04.994338 1368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:04.994374 1362 server_base.cc:1034] running on GCE node
W20260619 08:24:04.994292 1367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:04.994292 1370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:04.994789 1362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:04.995044 1362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:04.996215 1362 hybrid_clock.cc:648] HybridClock initialized: now 1781857444996184 us; error 47 us; skew 500 ppm
Jun 19 08:24:04 dist-test-slave-5rhg krb5kdc[1105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857444, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:24:04.998932 1362 init.cc:377] Logged in from keytab as oryx/127.25.127.130@KRBTEST.COM (short username oryx)
I20260619 08:24:04.999439 1362 webserver.cc:460] Webserver started at http://127.25.127.130:32891/ using document root <none> and password file <none>
I20260619 08:24:04.999787 1362 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:04.999856 1362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:04.999993 1362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:05.001175 1362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "a818706af2c4422b8dc86b89f502272b"
format_stamp: "Formatted at 2026-06-19 08:24:04 on dist-test-slave-5rhg"
server_key: "3a926ef23e4b06b7a0866aaa62a52102"
server_key_iv: "a8da041322e702faf9b08f4125d77236"
server_key_version: "encryptionkey@0"
I20260619 08:24:05.001540 1362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "a818706af2c4422b8dc86b89f502272b"
format_stamp: "Formatted at 2026-06-19 08:24:04 on dist-test-slave-5rhg"
server_key: "3a926ef23e4b06b7a0866aaa62a52102"
server_key_iv: "a8da041322e702faf9b08f4125d77236"
server_key_version: "encryptionkey@0"
I20260619 08:24:05.002887 1362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:24:05.003605 1377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:05.003803 1362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:24:05.003913 1362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "a818706af2c4422b8dc86b89f502272b"
format_stamp: "Formatted at 2026-06-19 08:24:04 on dist-test-slave-5rhg"
server_key: "3a926ef23e4b06b7a0866aaa62a52102"
server_key_iv: "a8da041322e702faf9b08f4125d77236"
server_key_version: "encryptionkey@0"
I20260619 08:24:05.003998 1362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:05.023734 1362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:05.026283 1362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:05.026463 1362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:05.027089 1362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:24:05.027151 1362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:05.027186 1362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:24:05.027215 1362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 08:24:05 dist-test-slave-5rhg krb5kdc[1105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857444, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM
I20260619 08:24:05.034523 1362 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:40921
I20260619 08:24:05.034668 1492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:40921 every 8 connection(s)
I20260619 08:24:05.034934 1362 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:24:05.037700 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.027511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:42589 (local address 127.25.127.190:38215)
0619 08:24:05.027582 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.027585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.027810 (+ 225us) server_negotiation.cc:407] Connection header received
0619 08:24:05.028405 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.028406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.028421 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.028501 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:05.029210 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.030064 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.031815 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.032203 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.032283 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.035002 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:05.035004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:05.035004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:05.035013 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:05.036472 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:05.036839 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:05.036840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:05.036840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:05.036892 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:05.037126 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:05.037128 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:05.037128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:05.037253 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:05.037319 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.037557 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.037634 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 08:24:05.037851 1386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.027505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:38215 (local address 127.25.127.130:42589)
0619 08:24:05.027665 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:05.027707 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:05.028298 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:05.028618 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:05.028626 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:05.028931 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:05.029136 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.029141 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.030177 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.030178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.031716 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.031721 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.032318 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.032319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.032463 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.033151 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:05.033161 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:05.034876 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:05.036548 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:05.036554 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:05.036556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:05.036775 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:05.036932 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:05.036932 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:05.036932 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:05.037041 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:05.037321 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:05.037327 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:05.037442 (+ 115us) client_negotiation.cc:769] Sending connection context
0619 08:24:05.037528 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 08:24:05.037651 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260619 08:24:05.038311 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 1362
I20260619 08:24:05.038341 1493 heartbeater.cc:347] Connected to a master server at 127.25.127.190:38215
I20260619 08:24:05.038432 1493 heartbeater.cc:464] Registering TS with master...
I20260619 08:24:05.038458 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:24:05.038640 1493 heartbeater.cc:511] Master 127.25.127.190:38215 requested a full tablet report, sending...
I20260619 08:24:05.038692 26110 external_mini_cluster.cc:1428] Setting key 10b844d814612c9d8aac4080488f0b28
I20260619 08:24:05.039034 1149 ts_manager.cc:194] Registered new tserver with Master: a818706af2c4422b8dc86b89f502272b (127.25.127.130:40921)
I20260619 08:24:05.039559 1149 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:42589
I20260619 08:24:05.046267 1499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.039420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:38215 (local address 127.25.127.130:42991)
0619 08:24:05.039544 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:05.039558 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:05.039612 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:05.039859 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:05.039860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:05.040096 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:05.040228 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.040230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.041057 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.041058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.041926 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.041931 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.042431 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.042431 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.042486 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.042978 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:05.042983 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:05.043302 (+ 319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:05.045199 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:05.045200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:05.045200 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:05.045368 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:05.045594 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:05.045594 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:05.045595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:05.045647 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:05.045999 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:05.045999 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:05.046045 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 08:24:05.046100 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 08:24:05.046181 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":10,"thread_start_us":65,"threads_started":1}
I20260619 08:24:05.046367 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.039703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:42991 (local address 127.25.127.190:38215)
0619 08:24:05.039743 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.039745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.039752 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:24:05.039773 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.039773 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.039784 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.039839 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:05.040308 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.040945 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.042037 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.042352 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.042424 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.043385 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:05.043386 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:05.043387 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:05.043395 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:05.045093 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:05.045456 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:05.045457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:05.045457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:05.045522 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:05.045712 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:05.045713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:05.045714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:05.045922 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:05.046007 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.046155 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.046242 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for oryx/127.25.127.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.131.keytab.
Entry for principal oryx/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.131.keytab.
I20260619 08:24:05.064700 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:38215
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.131.keytab
--principal=oryx/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:24:05.146533 1502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:05.146757 1502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:05.146785 1502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:05.147914 1502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:05.147948 1502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:05.147969 1502 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:24:05.147986 1502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:24:05.149492 1502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.131.keytab
--principal=oryx/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.127.190:38215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:05.149685 1502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:05.149866 1502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:05.154965 1510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:05.154986 1507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:05.155033 1508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:05.155198 1502 server_base.cc:1034] running on GCE node
I20260619 08:24:05.155476 1502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:05.155735 1502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:05.156973 1502 hybrid_clock.cc:648] HybridClock initialized: now 1781857445156937 us; error 57 us; skew 500 ppm
Jun 19 08:24:05 dist-test-slave-5rhg krb5kdc[1105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857445, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:24:05.159735 1502 init.cc:377] Logged in from keytab as oryx/127.25.127.131@KRBTEST.COM (short username oryx)
I20260619 08:24:05.160286 1502 webserver.cc:460] Webserver started at http://127.25.127.131:41383/ using document root <none> and password file <none>
I20260619 08:24:05.160524 1502 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:05.160593 1502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:05.160735 1502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:05.161816 1502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "16e493ae050f42d38bf36e60e629b78e"
format_stamp: "Formatted at 2026-06-19 08:24:05 on dist-test-slave-5rhg"
server_key: "6073204892a6bb2f73b923f6886c4a45"
server_key_iv: "110bb799d4e685b7d1afc85b7ecb92dc"
server_key_version: "encryptionkey@0"
I20260619 08:24:05.162200 1502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "16e493ae050f42d38bf36e60e629b78e"
format_stamp: "Formatted at 2026-06-19 08:24:05 on dist-test-slave-5rhg"
server_key: "6073204892a6bb2f73b923f6886c4a45"
server_key_iv: "110bb799d4e685b7d1afc85b7ecb92dc"
server_key_version: "encryptionkey@0"
I20260619 08:24:05.163584 1502 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:24:05.164398 1517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:05.164559 1502 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:05.164666 1502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "16e493ae050f42d38bf36e60e629b78e"
format_stamp: "Formatted at 2026-06-19 08:24:05 on dist-test-slave-5rhg"
server_key: "6073204892a6bb2f73b923f6886c4a45"
server_key_iv: "110bb799d4e685b7d1afc85b7ecb92dc"
server_key_version: "encryptionkey@0"
I20260619 08:24:05.164741 1502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:05.190740 1502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:05.193286 1502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:05.193457 1502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:05.194017 1502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:24:05.194090 1502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:05.194140 1502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:24:05.194173 1502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:05.201522 1502 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:45551
I20260619 08:24:05.201609 1632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:45551 every 8 connection(s)
I20260619 08:24:05.201944 1502 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:24:05 dist-test-slave-5rhg krb5kdc[1105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857445, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM
I20260619 08:24:05.205240 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.194649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:34813 (local address 127.25.127.190:38215)
0619 08:24:05.194705 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.194707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.194985 (+ 278us) server_negotiation.cc:407] Connection header received
0619 08:24:05.195694 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.195695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.195711 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.195774 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:05.196915 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.197775 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.199428 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.199700 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.199809 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.202583 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:05.202585 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:05.202585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:05.202594 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:05.203793 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:05.204238 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:05.204239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:05.204240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:05.204303 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:05.204571 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:05.204571 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:05.204572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:05.204694 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:05.204772 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.205055 (+ 283us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.205142 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:24:05.205426 1526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.194610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:38215 (local address 127.25.127.131:34813)
0619 08:24:05.194813 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:05.194886 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:05.195569 (+ 683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:05.195867 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:05.195872 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:05.196461 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:05.196795 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.196803 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.197908 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.197909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.199325 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.199329 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.199817 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.199818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.199966 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.200591 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:05.200601 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:05.202486 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:05.203880 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:05.203887 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:05.203890 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:05.204129 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:05.204375 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:05.204376 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:05.204377 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:05.204496 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:05.204762 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:05.204768 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:05.204916 (+ 148us) client_negotiation.cc:769] Sending connection context
0619 08:24:05.205019 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 08:24:05.205180 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260619 08:24:05.205858 1633 heartbeater.cc:347] Connected to a master server at 127.25.127.190:38215
I20260619 08:24:05.205955 1633 heartbeater.cc:464] Registering TS with master...
I20260619 08:24:05.206140 1633 heartbeater.cc:511] Master 127.25.127.190:38215 requested a full tablet report, sending...
I20260619 08:24:05.206532 1149 ts_manager.cc:194] Registered new tserver with Master: 16e493ae050f42d38bf36e60e629b78e (127.25.127.131:45551)
I20260619 08:24:05.207038 1149 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:34813
I20260619 08:24:05.209750 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 1502
I20260619 08:24:05.209899 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:24:05.210100 26110 external_mini_cluster.cc:1428] Setting key 4a590a62b88c9105599309dca246606f
I20260619 08:24:05.211117 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:24:05.213121 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.207095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:38573 (local address 127.25.127.190:38215)
0619 08:24:05.207136 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.207138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.207249 (+ 111us) server_negotiation.cc:407] Connection header received
0619 08:24:05.207310 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.207311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.207320 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.207387 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:05.207707 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.208302 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.209014 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.209224 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.209308 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.210418 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:05.210419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:05.210420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:05.210426 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:05.211894 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:05.212334 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:05.212335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:05.212335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:05.212384 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:05.212585 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:05.212585 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:05.212585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:05.212677 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:05.212723 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.212884 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.212942 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:24:05.213141 1639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.207050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:38215 (local address 127.25.127.131:38573)
0619 08:24:05.207177 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:05.207186 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:05.207263 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:05.207404 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:05.207405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:05.207566 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:05.207623 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.207624 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.208393 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.208394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.208912 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.208913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.209309 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.209310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.209385 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.209856 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:05.209861 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:05.210294 (+ 433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:05.211977 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:05.211978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:05.211979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:05.212267 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:05.212450 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:05.212451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:05.212452 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:05.212529 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:05.212734 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:05.212735 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:05.212788 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 08:24:05.212871 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 08:24:05.212994 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
Jun 19 08:24:05 dist-test-slave-5rhg krb5kdc[1105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857445, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 08:24:05 dist-test-slave-5rhg krb5kdc[1105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:05 dist-test-slave-5rhg krb5kdc[1105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 08:24:05.233840 1212 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:05.227838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49234 (local address 127.25.127.190:38215)
0619 08:24:05.227883 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.227885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.227895 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:24:05.227960 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.227962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.227971 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.228028 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:05.228719 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.229467 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.230334 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.230522 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.230585 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.233733 (+ 3148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49234: BlockingRecv error: recv got EOF from 127.0.0.1:49234 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 08:24:05 dist-test-slave-5rhg krb5kdc[1105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857445, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM
I20260619 08:24:05.241776 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.234613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49250 (local address 127.25.127.190:38215)
0619 08:24:05.234693 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.234695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.234786 (+ 91us) server_negotiation.cc:407] Connection header received
0619 08:24:05.234855 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.234856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.234874 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.234937 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:05.235418 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.235879 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.236663 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.236848 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.236952 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.239148 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:05.239149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:05.239149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:05.239155 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:05.240608 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:05.240946 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:05.240947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:05.240948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:05.241011 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:05.241215 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:05.241216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:05.241217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:05.241331 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:05.241447 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.241575 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.241657 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 08:24:05.243479 1149 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:24:05.245241 1149 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:24:05.251928 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.247828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:40921 (local address 127.0.0.1:38808)
0619 08:24:05.248198 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:05.248208 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:05.248259 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:05.248520 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:05.248521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:05.248523 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:05.248577 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.248579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.249463 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.249464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.250705 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.250707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.251629 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.251629 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.251710 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.251726 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 08:24:05.251791 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 08:24:05.251847 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":62,"threads_started":1}
I20260619 08:24:05.252002 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.247948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38808 (local address 127.25.127.130:40921)
0619 08:24:05.248131 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.248135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.248319 (+ 184us) server_negotiation.cc:407] Connection header received
0619 08:24:05.248366 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.248369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.248385 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.248458 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:05.248678 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.249369 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.250792 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.251537 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.251635 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.251695 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.251847 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.251896 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
I20260619 08:24:05.252694 1655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.247633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:43789 (local address 127.0.0.1:42498)
0619 08:24:05.247769 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:05.247813 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:05.247883 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:05.248381 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:05.248384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:05.248387 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:05.248463 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.248465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.249906 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.249907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.250926 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.250928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.252434 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.252434 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.252525 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.252530 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:24:05.252581 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 08:24:05.252626 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":48,"threads_started":1}
I20260619 08:24:05.252846 1427 tablet_service.cc:1467] Processing CreateTablet for tablet a525f83cc2b749b7a003ebc029228ccf (DEFAULT_TABLE table=test-table [id=1e95064a82c9490fa632228f0f992943]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:24:05.253036 1660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.247982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42498 (local address 127.25.127.129:43789)
0619 08:24:05.248195 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.248199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.248210 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:24:05.248254 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.248256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.248273 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.248539 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:05.248598 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.249762 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.251005 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.251723 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.251805 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.251855 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.252878 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.252922 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
I20260619 08:24:05.253082 1657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.247723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55170 (local address 127.25.127.131:45551)
0619 08:24:05.247989 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.247994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.248261 (+ 267us) server_negotiation.cc:407] Connection header received
0619 08:24:05.248378 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.248380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.248394 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.248447 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:05.248658 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.249765 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.251792 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.252609 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.252699 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.252749 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.252945 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.252985 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":93,"threads_started":1}
I20260619 08:24:05.253278 1427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a525f83cc2b749b7a003ebc029228ccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:05.253332 1658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.247641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:45551 (local address 127.0.0.1:55170)
0619 08:24:05.248139 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:05.248195 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:05.248287 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:05.248537 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:05.248537 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:05.248539 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:05.248584 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.248585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.250994 (+ 2409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.250994 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.251721 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.251722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.252783 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.252784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.252849 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.252853 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:24:05.253208 (+ 355us) client_negotiation.cc:240] Negotiation successful
0619 08:24:05.253256 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":90,"threads_started":1}
I20260619 08:24:05.253669 1287 tablet_service.cc:1467] Processing CreateTablet for tablet a525f83cc2b749b7a003ebc029228ccf (DEFAULT_TABLE table=test-table [id=1e95064a82c9490fa632228f0f992943]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:24:05.253971 1287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a525f83cc2b749b7a003ebc029228ccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:05.253953 1567 tablet_service.cc:1467] Processing CreateTablet for tablet a525f83cc2b749b7a003ebc029228ccf (DEFAULT_TABLE table=test-table [id=1e95064a82c9490fa632228f0f992943]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:24:05.254295 1567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a525f83cc2b749b7a003ebc029228ccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:05.257361 1661 tablet_bootstrap.cc:492] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662: Bootstrap starting.
I20260619 08:24:05.258036 1662 tablet_bootstrap.cc:492] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b: Bootstrap starting.
I20260619 08:24:05.258245 1661 tablet_bootstrap.cc:654] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:05.258301 1663 tablet_bootstrap.cc:492] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e: Bootstrap starting.
I20260619 08:24:05.258694 1662 tablet_bootstrap.cc:654] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:05.258704 1661 log.cc:824] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662: Log is configured to *not* fsync() on all Append() calls
I20260619 08:24:05.258814 1663 tablet_bootstrap.cc:654] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:05.259050 1662 log.cc:824] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b: Log is configured to *not* fsync() on all Append() calls
I20260619 08:24:05.259130 1663 log.cc:824] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e: Log is configured to *not* fsync() on all Append() calls
I20260619 08:24:05.259495 1661 tablet_bootstrap.cc:492] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662: No bootstrap required, opened a new log
I20260619 08:24:05.259552 1661 ts_tablet_manager.cc:1397] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:24:05.259797 1662 tablet_bootstrap.cc:492] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b: No bootstrap required, opened a new log
I20260619 08:24:05.259879 1662 ts_tablet_manager.cc:1397] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:24:05.260838 1663 tablet_bootstrap.cc:492] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e: No bootstrap required, opened a new log
I20260619 08:24:05.260908 1663 ts_tablet_manager.cc:1397] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 08:24:05.261600 1662 raft_consensus.cc:348] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } }
I20260619 08:24:05.261766 1662 raft_consensus.cc:374] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:05.261811 1662 raft_consensus.cc:729] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a818706af2c4422b8dc86b89f502272b, State: Initialized, Role: FOLLOWER
I20260619 08:24:05.261687 1661 raft_consensus.cc:348] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } }
I20260619 08:24:05.261852 1661 raft_consensus.cc:374] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:05.261895 1661 raft_consensus.cc:729] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a39e357cd3814cd69a9a5b91fb41b662, State: Initialized, Role: FOLLOWER
I20260619 08:24:05.261929 1662 consensus_queue.cc:260] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } }
I20260619 08:24:05.262004 1661 consensus_queue.cc:260] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } }
I20260619 08:24:05.262239 1662 ts_tablet_manager.cc:1428] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 08:24:05.262267 1661 ts_tablet_manager.cc:1428] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:24:05.262387 1493 heartbeater.cc:503] Master 127.25.127.190:38215 was elected leader, sending a full tablet report...
I20260619 08:24:05.262382 1353 heartbeater.cc:503] Master 127.25.127.190:38215 was elected leader, sending a full tablet report...
I20260619 08:24:05.262995 1663 raft_consensus.cc:348] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } }
I20260619 08:24:05.263139 1663 raft_consensus.cc:374] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:05.263182 1663 raft_consensus.cc:729] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16e493ae050f42d38bf36e60e629b78e, State: Initialized, Role: FOLLOWER
I20260619 08:24:05.263300 1663 consensus_queue.cc:260] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } }
I20260619 08:24:05.263594 1663 ts_tablet_manager.cc:1428] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:24:05.263641 1633 heartbeater.cc:503] Master 127.25.127.190:38215 was elected leader, sending a full tablet report...
W20260619 08:24:05.285823 1494 tablet.cc:2369] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:24:05.311905 1667 raft_consensus.cc:482] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:24:05.312031 1667 raft_consensus.cc:504] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } }
I20260619 08:24:05.312479 1667 leader_election.cc:288] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16e493ae050f42d38bf36e60e629b78e (127.25.127.131:45551), a39e357cd3814cd69a9a5b91fb41b662 (127.25.127.129:43789)
I20260619 08:24:05.315428 1386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.312678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:43789 (local address 127.25.127.130:43893)
0619 08:24:05.312742 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:05.312757 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:05.312807 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:05.313004 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:05.313005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:05.313007 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:05.313109 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.313112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.313816 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.313818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.314597 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.314599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.315142 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.315143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.315227 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.315230 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:24:05.315297 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 08:24:05.315338 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 08:24:05.315413 1660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.312742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:43893 (local address 127.25.127.129:43789)
0619 08:24:05.312796 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.312798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.312807 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:24:05.312891 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.312893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.312904 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.312972 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:05.313239 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.313723 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.314693 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.315052 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.315134 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.315143 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.315303 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.315339 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:24:05.315759 1307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a525f83cc2b749b7a003ebc029228ccf" candidate_uuid: "a818706af2c4422b8dc86b89f502272b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a39e357cd3814cd69a9a5b91fb41b662" is_pre_election: true
I20260619 08:24:05.315867 1657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.312730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:54737 (local address 127.25.127.131:45551)
0619 08:24:05.312779 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.312782 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.313109 (+ 327us) server_negotiation.cc:407] Connection header received
0619 08:24:05.313151 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.313152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.313159 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.313199 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:05.313338 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.313806 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.314719 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.315086 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.315166 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.315174 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.315761 (+ 587us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.315801 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:24:05.315889 1670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.312681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:45551 (local address 127.25.127.130:54737)
0619 08:24:05.313032 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:05.313042 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:05.313100 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:05.313203 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:05.313203 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:05.313204 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:05.313264 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.313265 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.313890 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.313891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.314628 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.314630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.315167 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.315167 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.315671 (+ 504us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.315681 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 08:24:05.315755 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 08:24:05.315795 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":134,"thread_start_us":137,"threads_started":1}
I20260619 08:24:05.315943 1307 raft_consensus.cc:2454] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a818706af2c4422b8dc86b89f502272b in term 0.
I20260619 08:24:05.316239 1378 leader_election.cc:302] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a39e357cd3814cd69a9a5b91fb41b662, a818706af2c4422b8dc86b89f502272b; no voters:
I20260619 08:24:05.316355 1587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a525f83cc2b749b7a003ebc029228ccf" candidate_uuid: "a818706af2c4422b8dc86b89f502272b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16e493ae050f42d38bf36e60e629b78e" is_pre_election: true
I20260619 08:24:05.316395 1667 raft_consensus.cc:2781] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:24:05.316464 1667 raft_consensus.cc:482] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:24:05.316491 1667 raft_consensus.cc:3037] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:05.316525 1587 raft_consensus.cc:2454] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a818706af2c4422b8dc86b89f502272b in term 0.
I20260619 08:24:05.317250 1667 raft_consensus.cc:504] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } }
I20260619 08:24:05.317404 1667 leader_election.cc:288] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [CANDIDATE]: Term 1 election: Requested vote from peers 16e493ae050f42d38bf36e60e629b78e (127.25.127.131:45551), a39e357cd3814cd69a9a5b91fb41b662 (127.25.127.129:43789)
I20260619 08:24:05.317548 1587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a525f83cc2b749b7a003ebc029228ccf" candidate_uuid: "a818706af2c4422b8dc86b89f502272b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16e493ae050f42d38bf36e60e629b78e"
I20260619 08:24:05.317590 1307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a525f83cc2b749b7a003ebc029228ccf" candidate_uuid: "a818706af2c4422b8dc86b89f502272b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a39e357cd3814cd69a9a5b91fb41b662"
I20260619 08:24:05.317633 1587 raft_consensus.cc:3037] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:05.317655 1307 raft_consensus.cc:3037] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:05.318490 1307 raft_consensus.cc:2454] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a818706af2c4422b8dc86b89f502272b in term 1.
I20260619 08:24:05.318490 1587 raft_consensus.cc:2454] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a818706af2c4422b8dc86b89f502272b in term 1.
I20260619 08:24:05.318665 1380 leader_election.cc:302] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16e493ae050f42d38bf36e60e629b78e, a818706af2c4422b8dc86b89f502272b; no voters:
I20260619 08:24:05.318768 1667 raft_consensus.cc:2781] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:24:05.318919 1667 raft_consensus.cc:686] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [term 1 LEADER]: Becoming Leader. State: Replica: a818706af2c4422b8dc86b89f502272b, State: Running, Role: LEADER
I20260619 08:24:05.319023 1667 consensus_queue.cc:237] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } }
I20260619 08:24:05.319870 1150 catalog_manager.cc:5495] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b reported cstate change: term changed from 0 to 1, leader changed from <none> to a818706af2c4422b8dc86b89f502272b (127.25.127.130). New cstate: current_term: 1 leader_uuid: "a818706af2c4422b8dc86b89f502272b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } health_report { overall_health: HEALTHY } } }
I20260619 08:24:05.329645 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.327423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38824 (local address 127.25.127.130:40921)
0619 08:24:05.327467 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.327469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.327476 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:24:05.327551 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.327552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.327566 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.327622 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:24:05.327776 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.328288 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.328988 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.329174 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.329221 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.329353 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:24:05.329439 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:24:05.329490 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.329539 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.329554 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:24:05.331430 1587 raft_consensus.cc:1261] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e [term 1 FOLLOWER]: Refusing update from remote peer a818706af2c4422b8dc86b89f502272b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:24:05.331434 1307 raft_consensus.cc:1261] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662 [term 1 FOLLOWER]: Refusing update from remote peer a818706af2c4422b8dc86b89f502272b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:24:05.331790 1671 consensus_queue.cc:1035] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [LEADER]: Connected to new peer: Peer: permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:24:05.331951 1667 consensus_queue.cc:1035] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:24:05.334398 1674 mvcc.cc:205] Tried to move back new op lower bound from 7298488096074555392 to 7298488096027344896. Current Snapshot: MvccSnapshot[applied={T|T < 7298488096074555392}]
I20260619 08:24:05.334672 1675 mvcc.cc:205] Tried to move back new op lower bound from 7298488096074555392 to 7298488096027344896. Current Snapshot: MvccSnapshot[applied={T|T < 7298488096074555392}]
I20260619 08:24:05.335074 1676 mvcc.cc:205] Tried to move back new op lower bound from 7298488096074555392 to 7298488096027344896. Current Snapshot: MvccSnapshot[applied={T|T < 7298488096074555392}]
I20260619 08:24:05.336911 1149 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:24:05.337000 1149 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:24:05.337730 1149 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932: Sending DeleteTablet for 3 replicas of tablet a525f83cc2b749b7a003ebc029228ccf
I20260619 08:24:05.338048 1287 tablet_service.cc:1514] Processing DeleteTablet for tablet a525f83cc2b749b7a003ebc029228ccf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:05 UTC) from {username='oryx', principal='oryx/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:42498
I20260619 08:24:05.338048 1567 tablet_service.cc:1514] Processing DeleteTablet for tablet a525f83cc2b749b7a003ebc029228ccf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:05 UTC) from {username='oryx', principal='oryx/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:55170
I20260619 08:24:05.338085 1427 tablet_service.cc:1514] Processing DeleteTablet for tablet a525f83cc2b749b7a003ebc029228ccf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:05 UTC) from {username='oryx', principal='oryx/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:38808
I20260619 08:24:05.338357 1684 tablet_replica.cc:330] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b: stopping tablet replica
I20260619 08:24:05.338340 1683 tablet_replica.cc:330] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662: stopping tablet replica
I20260619 08:24:05.338493 1684 raft_consensus.cc:2229] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:24:05.338522 1149 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 08:24:05.338615 1684 raft_consensus.cc:2258] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 08:24:05.338666 1149 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:24:05.338838 1683 raft_consensus.cc:2229] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:24:05.339560 1683 raft_consensus.cc:2258] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:24:05.339742 1684 ts_tablet_manager.cc:1905] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:05.339934 1287 tablet_service.cc:1467] Processing CreateTablet for tablet 332197f981744fba96d26fe60c7bbd65 (DEFAULT_TABLE table=test-table [id=9808776685054a5fb13c475914087959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:24:05.339968 1427 tablet_service.cc:1467] Processing CreateTablet for tablet 332197f981744fba96d26fe60c7bbd65 (DEFAULT_TABLE table=test-table [id=9808776685054a5fb13c475914087959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:24:05.340054 1427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 332197f981744fba96d26fe60c7bbd65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:05.340054 1287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 332197f981744fba96d26fe60c7bbd65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:05.340351 1567 tablet_service.cc:1467] Processing CreateTablet for tablet 332197f981744fba96d26fe60c7bbd65 (DEFAULT_TABLE table=test-table [id=9808776685054a5fb13c475914087959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:24:05.340453 1567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 332197f981744fba96d26fe60c7bbd65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:05.341655 1684 ts_tablet_manager.cc:1918] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:24:05.341732 1684 log.cc:1196] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/a525f83cc2b749b7a003ebc029228ccf
I20260619 08:24:05.341970 1663 tablet_bootstrap.cc:492] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e: Bootstrap starting.
I20260619 08:24:05.342002 1661 tablet_bootstrap.cc:492] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662: Bootstrap starting.
I20260619 08:24:05.342011 1684 ts_tablet_manager.cc:1939] T a525f83cc2b749b7a003ebc029228ccf P a818706af2c4422b8dc86b89f502272b: Deleting consensus metadata
I20260619 08:24:05.342386 1134 catalog_manager.cc:4841] TS a818706af2c4422b8dc86b89f502272b (127.25.127.130:40921): tablet a525f83cc2b749b7a003ebc029228ccf (table test-table [id=1e95064a82c9490fa632228f0f992943]) successfully deleted
I20260619 08:24:05.342434 1661 tablet_bootstrap.cc:654] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:05.342434 1663 tablet_bootstrap.cc:654] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:05.342569 1685 tablet_replica.cc:330] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e: stopping tablet replica
I20260619 08:24:05.342674 1685 raft_consensus.cc:2229] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:24:05.342774 1685 raft_consensus.cc:2258] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:24:05.343326 1661 tablet_bootstrap.cc:492] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662: No bootstrap required, opened a new log
I20260619 08:24:05.343375 1685 ts_tablet_manager.cc:1905] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:05.343390 1661 ts_tablet_manager.cc:1397] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:05.343545 1661 raft_consensus.cc:348] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.343636 1661 raft_consensus.cc:374] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:05.343662 1661 raft_consensus.cc:729] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a39e357cd3814cd69a9a5b91fb41b662, State: Initialized, Role: FOLLOWER
I20260619 08:24:05.343713 1661 consensus_queue.cc:260] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.343801 1661 ts_tablet_manager.cc:1428] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:05.344172 1662 tablet_bootstrap.cc:492] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b: Bootstrap starting.
I20260619 08:24:05.344633 1662 tablet_bootstrap.cc:654] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:05.345110 1685 ts_tablet_manager.cc:1918] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:24:05.345193 1685 log.cc:1196] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/a525f83cc2b749b7a003ebc029228ccf
I20260619 08:24:05.345278 1662 tablet_bootstrap.cc:492] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b: No bootstrap required, opened a new log
I20260619 08:24:05.345319 1662 ts_tablet_manager.cc:1397] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:05.345306 1683 ts_tablet_manager.cc:1905] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:05.345443 1685 ts_tablet_manager.cc:1939] T a525f83cc2b749b7a003ebc029228ccf P 16e493ae050f42d38bf36e60e629b78e: Deleting consensus metadata
I20260619 08:24:05.345485 1662 raft_consensus.cc:348] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.345551 1662 raft_consensus.cc:374] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:05.345582 1662 raft_consensus.cc:729] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a818706af2c4422b8dc86b89f502272b, State: Initialized, Role: FOLLOWER
I20260619 08:24:05.345625 1662 consensus_queue.cc:260] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.345710 1662 ts_tablet_manager.cc:1428] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:24:05.345860 1136 catalog_manager.cc:4841] TS 16e493ae050f42d38bf36e60e629b78e (127.25.127.131:45551): tablet a525f83cc2b749b7a003ebc029228ccf (table test-table [id=1e95064a82c9490fa632228f0f992943]) successfully deleted
I20260619 08:24:05.347039 1683 ts_tablet_manager.cc:1918] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:24:05.347129 1683 log.cc:1196] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/wal/wals/a525f83cc2b749b7a003ebc029228ccf
I20260619 08:24:05.347363 1683 ts_tablet_manager.cc:1939] T a525f83cc2b749b7a003ebc029228ccf P a39e357cd3814cd69a9a5b91fb41b662: Deleting consensus metadata
I20260619 08:24:05.347746 1134 catalog_manager.cc:4841] TS a39e357cd3814cd69a9a5b91fb41b662 (127.25.127.129:43789): tablet a525f83cc2b749b7a003ebc029228ccf (table test-table [id=1e95064a82c9490fa632228f0f992943]) successfully deleted
I20260619 08:24:05.352226 1663 tablet_bootstrap.cc:492] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e: No bootstrap required, opened a new log
I20260619 08:24:05.352309 1663 ts_tablet_manager.cc:1397] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260619 08:24:05.352483 1663 raft_consensus.cc:348] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.352571 1663 raft_consensus.cc:374] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:05.352595 1663 raft_consensus.cc:729] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16e493ae050f42d38bf36e60e629b78e, State: Initialized, Role: FOLLOWER
I20260619 08:24:05.352648 1663 consensus_queue.cc:260] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.352751 1663 ts_tablet_manager.cc:1428] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260619 08:24:05.375855 1354 tablet.cc:2369] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:24:05.452499 1634 tablet.cc:2369] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:24:05.607699 1669 raft_consensus.cc:482] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:24:05.607806 1669 raft_consensus.cc:504] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.608259 1669 leader_election.cc:288] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a39e357cd3814cd69a9a5b91fb41b662 (127.25.127.129:43789), a818706af2c4422b8dc86b89f502272b (127.25.127.130:40921)
I20260619 08:24:05.611361 1660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.608558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:49003 (local address 127.25.127.129:43789)
0619 08:24:05.608602 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.608604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.608616 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:24:05.608654 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.608655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.608662 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.608755 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:05.608974 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.609525 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.610600 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.610982 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.611039 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.611046 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.611241 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.611286 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:24:05.611416 1526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.608473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:43789 (local address 127.25.127.131:49003)
0619 08:24:05.608537 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:05.608551 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:05.608602 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:05.608762 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:05.608763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:05.608766 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:05.608848 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.608850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.609631 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.609632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.610505 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.610508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.611072 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.611073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.611161 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.611165 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:24:05.611226 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 08:24:05.611274 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 08:24:05.611443 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.608733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:55867 (local address 127.25.127.130:40921)
0619 08:24:05.608778 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.608780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.608788 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:24:05.608846 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.608847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.608861 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.608943 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:05.609156 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.609692 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.610654 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.611073 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.611155 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.611164 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.611325 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.611367 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:24:05.611531 1689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.608600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:40921 (local address 127.25.127.131:55867)
0619 08:24:05.608732 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:05.608743 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:05.608793 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:05.608960 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:05.608961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:05.608963 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:05.609056 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.609058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.609785 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.609786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.610519 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.610520 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.611163 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.611164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.611243 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.611247 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:24:05.611304 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 08:24:05.611341 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":12,"thread_start_us":73,"threads_started":1}
I20260619 08:24:05.611712 1307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "332197f981744fba96d26fe60c7bbd65" candidate_uuid: "16e493ae050f42d38bf36e60e629b78e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a39e357cd3814cd69a9a5b91fb41b662" is_pre_election: true
I20260619 08:24:05.611829 1307 raft_consensus.cc:2454] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16e493ae050f42d38bf36e60e629b78e in term 0.
I20260619 08:24:05.611817 1447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "332197f981744fba96d26fe60c7bbd65" candidate_uuid: "16e493ae050f42d38bf36e60e629b78e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a818706af2c4422b8dc86b89f502272b" is_pre_election: true
I20260619 08:24:05.611968 1447 raft_consensus.cc:2454] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16e493ae050f42d38bf36e60e629b78e in term 0.
I20260619 08:24:05.612025 1518 leader_election.cc:302] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16e493ae050f42d38bf36e60e629b78e, a39e357cd3814cd69a9a5b91fb41b662; no voters:
I20260619 08:24:05.612179 1669 raft_consensus.cc:2781] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:24:05.612222 1669 raft_consensus.cc:482] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:24:05.612246 1669 raft_consensus.cc:3037] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:05.612919 1669 raft_consensus.cc:504] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.613052 1669 leader_election.cc:288] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [CANDIDATE]: Term 1 election: Requested vote from peers a39e357cd3814cd69a9a5b91fb41b662 (127.25.127.129:43789), a818706af2c4422b8dc86b89f502272b (127.25.127.130:40921)
I20260619 08:24:05.613235 1447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "332197f981744fba96d26fe60c7bbd65" candidate_uuid: "16e493ae050f42d38bf36e60e629b78e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a818706af2c4422b8dc86b89f502272b"
I20260619 08:24:05.613235 1307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "332197f981744fba96d26fe60c7bbd65" candidate_uuid: "16e493ae050f42d38bf36e60e629b78e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a39e357cd3814cd69a9a5b91fb41b662"
I20260619 08:24:05.613322 1447 raft_consensus.cc:3037] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:05.613322 1307 raft_consensus.cc:3037] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:05.614099 1307 raft_consensus.cc:2454] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16e493ae050f42d38bf36e60e629b78e in term 1.
I20260619 08:24:05.614214 1447 raft_consensus.cc:2454] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16e493ae050f42d38bf36e60e629b78e in term 1.
I20260619 08:24:05.614266 1518 leader_election.cc:302] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16e493ae050f42d38bf36e60e629b78e, a39e357cd3814cd69a9a5b91fb41b662; no voters:
I20260619 08:24:05.614352 1669 raft_consensus.cc:2781] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:24:05.614518 1669 raft_consensus.cc:686] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [term 1 LEADER]: Becoming Leader. State: Replica: 16e493ae050f42d38bf36e60e629b78e, State: Running, Role: LEADER
I20260619 08:24:05.614603 1669 consensus_queue.cc:237] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.615386 1150 catalog_manager.cc:5495] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e reported cstate change: term changed from 0 to 1, leader changed from <none> to 16e493ae050f42d38bf36e60e629b78e (127.25.127.131). New cstate: current_term: 1 leader_uuid: "16e493ae050f42d38bf36e60e629b78e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } health_report { overall_health: HEALTHY } } }
Jun 19 08:24:05 dist-test-slave-5rhg krb5kdc[1105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857444, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.190@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM
I20260619 08:24:05.651991 1697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.644986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:38215 (local address 127.0.0.1:49260)
0619 08:24:05.645156 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:05.645169 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:05.645240 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:05.645388 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:05.645389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:05.645700 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:05.645755 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.645757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.646399 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.646399 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.646965 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.646967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.647303 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.647305 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.647353 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.647918 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:05.647923 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:05.649330 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:05.650936 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:05.650945 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:05.650947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:05.651159 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:05.651345 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:05.651346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:05.651346 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:05.651426 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:05.651673 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:05.651680 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:05.651724 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 08:24:05.651787 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 08:24:05.651883 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":103,"threads_started":1}
I20260619 08:24:05.652035 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.645045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49260 (local address 127.25.127.190:38215)
0619 08:24:05.645118 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.645124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.645240 (+ 116us) server_negotiation.cc:407] Connection header received
0619 08:24:05.645305 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.645306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.645315 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.645360 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:05.645822 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.646316 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.647045 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.647220 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.647272 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.649417 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:05.649418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:05.649419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:05.649424 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:05.650850 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:05.651232 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:05.651233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:05.651233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:05.651283 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:05.651485 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:05.651486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:05.651486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:05.651591 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:05.651641 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.651818 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.651927 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":46}
I20260619 08:24:05.654227 1150 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:49260:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 08:24:05.654404 1150 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:24:05.655926 1287 tablet_service.cc:1467] Processing CreateTablet for tablet 7851a76716d04baeb2fe83befa482e6c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=97f218d754584f4187db5f877c7062b6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:24:05.655958 1427 tablet_service.cc:1467] Processing CreateTablet for tablet 7851a76716d04baeb2fe83befa482e6c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=97f218d754584f4187db5f877c7062b6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:24:05.655956 1567 tablet_service.cc:1467] Processing CreateTablet for tablet 7851a76716d04baeb2fe83befa482e6c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=97f218d754584f4187db5f877c7062b6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:24:05.656100 1287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7851a76716d04baeb2fe83befa482e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:05.656100 1427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7851a76716d04baeb2fe83befa482e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:05.656113 1567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7851a76716d04baeb2fe83befa482e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:05.657529 1663 tablet_bootstrap.cc:492] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e: Bootstrap starting.
I20260619 08:24:05.657692 1662 tablet_bootstrap.cc:492] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b: Bootstrap starting.
I20260619 08:24:05.657984 1663 tablet_bootstrap.cc:654] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:05.658068 1662 tablet_bootstrap.cc:654] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:05.658314 1661 tablet_bootstrap.cc:492] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662: Bootstrap starting.
I20260619 08:24:05.658640 1663 tablet_bootstrap.cc:492] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e: No bootstrap required, opened a new log
I20260619 08:24:05.658679 1661 tablet_bootstrap.cc:654] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:05.658708 1663 ts_tablet_manager.cc:1397] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:05.659089 1663 raft_consensus.cc:348] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.659186 1663 raft_consensus.cc:374] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:05.659214 1663 raft_consensus.cc:729] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16e493ae050f42d38bf36e60e629b78e, State: Initialized, Role: FOLLOWER
I20260619 08:24:05.659214 1662 tablet_bootstrap.cc:492] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b: No bootstrap required, opened a new log
I20260619 08:24:05.659255 1662 ts_tablet_manager.cc:1397] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260619 08:24:05.659276 1661 tablet_bootstrap.cc:492] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662: No bootstrap required, opened a new log
I20260619 08:24:05.659274 1663 consensus_queue.cc:260] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.659312 1661 ts_tablet_manager.cc:1397] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:05.659385 1663 ts_tablet_manager.cc:1428] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:05.659387 1662 raft_consensus.cc:348] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.659466 1662 raft_consensus.cc:374] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:05.659493 1662 raft_consensus.cc:729] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a818706af2c4422b8dc86b89f502272b, State: Initialized, Role: FOLLOWER
I20260619 08:24:05.659461 1661 raft_consensus.cc:348] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.659538 1661 raft_consensus.cc:374] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:05.659560 1661 raft_consensus.cc:729] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a39e357cd3814cd69a9a5b91fb41b662, State: Initialized, Role: FOLLOWER
I20260619 08:24:05.659549 1662 consensus_queue.cc:260] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.659539 1669 tablet_replica.cc:439] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } }
I20260619 08:24:05.659610 1669 tablet_replica.cc:442] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:24:05.659605 1661 consensus_queue.cc:260] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.659641 1662 ts_tablet_manager.cc:1428] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:05.659688 1661 ts_tablet_manager.cc:1428] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:05.659737 1668 tablet_replica.cc:439] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } }
I20260619 08:24:05.659801 1668 tablet_replica.cc:442] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:24:05.659749 1671 tablet_replica.cc:439] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } }
I20260619 08:24:05.659832 1671 tablet_replica.cc:442] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:24:05.690564 1668 raft_consensus.cc:482] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:24:05.690682 1668 raft_consensus.cc:504] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.691025 1668 leader_election.cc:288] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a818706af2c4422b8dc86b89f502272b (127.25.127.130:40921), 16e493ae050f42d38bf36e60e629b78e (127.25.127.131:45551)
I20260619 08:24:05.695096 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.691282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:59897 (local address 127.25.127.130:40921)
0619 08:24:05.691337 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.691340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.691446 (+ 106us) server_negotiation.cc:407] Connection header received
0619 08:24:05.691488 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.691489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.691500 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.691574 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:05.691776 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.692563 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.694134 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.694721 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.694793 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.694802 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.694995 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.695034 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:24:05.695093 1657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.691295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:34121 (local address 127.25.127.131:45551)
0619 08:24:05.691341 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.691343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.691447 (+ 104us) server_negotiation.cc:407] Connection header received
0619 08:24:05.691492 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.691493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.691501 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.691574 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:05.691775 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.692566 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.694136 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.694715 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.694793 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.694802 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.694977 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.695023 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:24:05.695133 1701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.691208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:40921 (local address 127.25.127.129:59897)
0619 08:24:05.691374 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:05.691385 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:05.691434 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:05.691593 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:05.691594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:05.691596 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:05.691689 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.691691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.692653 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.692654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.694059 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.694061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.694811 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.694812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.694885 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.694890 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:24:05.694954 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 08:24:05.695039 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":83,"spinlock_wait_cycles":896,"thread_start_us":74,"threads_started":1}
I20260619 08:24:05.695133 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.691208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:45551 (local address 127.25.127.129:34121)
0619 08:24:05.691351 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:05.691364 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:05.691434 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:05.691593 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:05.691594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:05.691597 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:05.691706 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.691708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.692654 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.692655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.694043 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.694047 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.694810 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.694811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.694889 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.694891 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:24:05.694953 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 08:24:05.695016 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":109,"thread_start_us":77,"threads_started":1}
I20260619 08:24:05.695422 1587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7851a76716d04baeb2fe83befa482e6c" candidate_uuid: "a39e357cd3814cd69a9a5b91fb41b662" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16e493ae050f42d38bf36e60e629b78e" is_pre_election: true
I20260619 08:24:05.695477 1447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7851a76716d04baeb2fe83befa482e6c" candidate_uuid: "a39e357cd3814cd69a9a5b91fb41b662" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a818706af2c4422b8dc86b89f502272b" is_pre_election: true
I20260619 08:24:05.695533 1587 raft_consensus.cc:2454] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a39e357cd3814cd69a9a5b91fb41b662 in term 0.
I20260619 08:24:05.695571 1447 raft_consensus.cc:2454] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a39e357cd3814cd69a9a5b91fb41b662 in term 0.
I20260619 08:24:05.695708 1240 leader_election.cc:302] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16e493ae050f42d38bf36e60e629b78e, a39e357cd3814cd69a9a5b91fb41b662; no voters:
I20260619 08:24:05.695868 1668 raft_consensus.cc:2781] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:24:05.695923 1668 raft_consensus.cc:482] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:24:05.695945 1668 raft_consensus.cc:3037] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:05.696707 1668 raft_consensus.cc:504] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.696853 1668 leader_election.cc:288] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [CANDIDATE]: Term 1 election: Requested vote from peers a818706af2c4422b8dc86b89f502272b (127.25.127.130:40921), 16e493ae050f42d38bf36e60e629b78e (127.25.127.131:45551)
I20260619 08:24:05.697011 1447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7851a76716d04baeb2fe83befa482e6c" candidate_uuid: "a39e357cd3814cd69a9a5b91fb41b662" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a818706af2c4422b8dc86b89f502272b"
I20260619 08:24:05.697041 1587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7851a76716d04baeb2fe83befa482e6c" candidate_uuid: "a39e357cd3814cd69a9a5b91fb41b662" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16e493ae050f42d38bf36e60e629b78e"
I20260619 08:24:05.697091 1447 raft_consensus.cc:3037] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:05.697098 1587 raft_consensus.cc:3037] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:05.697856 1447 raft_consensus.cc:2454] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a39e357cd3814cd69a9a5b91fb41b662 in term 1.
I20260619 08:24:05.697854 1587 raft_consensus.cc:2454] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a39e357cd3814cd69a9a5b91fb41b662 in term 1.
I20260619 08:24:05.698057 1238 leader_election.cc:302] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a39e357cd3814cd69a9a5b91fb41b662, a818706af2c4422b8dc86b89f502272b; no voters:
I20260619 08:24:05.698181 1668 raft_consensus.cc:2781] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:24:05.698310 1668 raft_consensus.cc:686] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [term 1 LEADER]: Becoming Leader. State: Replica: a39e357cd3814cd69a9a5b91fb41b662, State: Running, Role: LEADER
I20260619 08:24:05.698379 1668 consensus_queue.cc:237] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.698756 1703 tablet_replica.cc:439] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662: TxnStatusTablet state changed. Reason: New leader a39e357cd3814cd69a9a5b91fb41b662. Latest consensus state: current_term: 1 leader_uuid: "a39e357cd3814cd69a9a5b91fb41b662" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } }
I20260619 08:24:05.698836 1703 tablet_replica.cc:442] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:24:05.698997 1705 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 08:24:05.699150 1705 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 08:24:05.699149 1148 catalog_manager.cc:5495] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 reported cstate change: term changed from 0 to 1, leader changed from <none> to a39e357cd3814cd69a9a5b91fb41b662 (127.25.127.129). New cstate: current_term: 1 leader_uuid: "a39e357cd3814cd69a9a5b91fb41b662" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } health_report { overall_health: UNKNOWN } } }
I20260619 08:24:05.706991 1660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.704217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42500 (local address 127.25.127.129:43789)
0619 08:24:05.704256 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.704257 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.704273 (+ 16us) server_negotiation.cc:407] Connection header received
0619 08:24:05.704367 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.704368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.704384 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.704440 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:24:05.704598 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.705048 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.706144 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.706333 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.706380 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.706602 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:24:05.706754 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:24:05.706844 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.706918 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.706937 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:24:05.707059 1697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.704120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:43789 (local address 127.0.0.1:42500)
0619 08:24:05.704228 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:05.704241 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:05.704288 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:05.704466 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:05.704467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:05.704468 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:24:05.704534 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.704537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.705126 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.705127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.706042 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.706045 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.706405 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.706406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.706482 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.706499 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:24:05.706842 (+ 343us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:24:05.706851 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 08:24:05.706912 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 08:24:05.706960 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260619 08:24:05.708081 1587 raft_consensus.cc:1261] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e [term 1 FOLLOWER]: Refusing update from remote peer a39e357cd3814cd69a9a5b91fb41b662: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:24:05.708133 1447 raft_consensus.cc:1261] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b [term 1 FOLLOWER]: Refusing update from remote peer a39e357cd3814cd69a9a5b91fb41b662: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:24:05.708367 1668 consensus_queue.cc:1035] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:24:05.708501 1703 consensus_queue.cc:1035] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:24:05.709174 1671 tablet_replica.cc:439] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b: TxnStatusTablet state changed. Reason: New leader a39e357cd3814cd69a9a5b91fb41b662. Latest consensus state: current_term: 1 leader_uuid: "a39e357cd3814cd69a9a5b91fb41b662" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } }
I20260619 08:24:05.709271 1671 tablet_replica.cc:442] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:24:05.709266 1669 tablet_replica.cc:439] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e: TxnStatusTablet state changed. Reason: New leader a39e357cd3814cd69a9a5b91fb41b662. Latest consensus state: current_term: 1 leader_uuid: "a39e357cd3814cd69a9a5b91fb41b662" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } }
I20260619 08:24:05.709336 1669 tablet_replica.cc:442] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:24:05.710088 1703 tablet_replica.cc:439] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a39e357cd3814cd69a9a5b91fb41b662" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } }
I20260619 08:24:05.710168 1703 tablet_replica.cc:442] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:24:05.710414 1671 tablet_replica.cc:439] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a39e357cd3814cd69a9a5b91fb41b662" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } }
I20260619 08:24:05.710502 1671 tablet_replica.cc:442] T 7851a76716d04baeb2fe83befa482e6c P a818706af2c4422b8dc86b89f502272b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:24:05.710772 1669 tablet_replica.cc:439] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a39e357cd3814cd69a9a5b91fb41b662" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } }
I20260619 08:24:05.710856 1669 tablet_replica.cc:442] T 7851a76716d04baeb2fe83befa482e6c P 16e493ae050f42d38bf36e60e629b78e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:24:05.711540 1710 tablet_replica.cc:439] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a39e357cd3814cd69a9a5b91fb41b662" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } } }
I20260619 08:24:05.711642 1710 tablet_replica.cc:442] T 7851a76716d04baeb2fe83befa482e6c P a39e357cd3814cd69a9a5b91fb41b662: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:24:05.714534 1657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.712089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55186 (local address 127.25.127.131:45551)
0619 08:24:05.712133 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.712135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.712163 (+ 28us) server_negotiation.cc:407] Connection header received
0619 08:24:05.712245 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.712245 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.712267 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.712350 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:24:05.712458 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.712927 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.713738 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.713948 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.714026 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.714201 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:24:05.714311 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:24:05.714395 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.714421 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.714443 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:24:05.719506 1660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.716903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:43597 (local address 127.25.127.129:43789)
0619 08:24:05.716950 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.716951 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.717061 (+ 110us) server_negotiation.cc:407] Connection header received
0619 08:24:05.717137 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.717138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.717146 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.717190 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:24:05.717350 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.717788 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.718860 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.719047 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.719096 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.719292 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:24:05.719317 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:24:05.719355 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.719435 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.719452 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:24:05.719661 1714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.716806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:43789 (local address 127.25.127.131:43597)
0619 08:24:05.716987 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:05.716999 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:05.717071 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:05.717211 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:05.717212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:05.717214 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:24:05.717277 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.717278 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.717857 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.717858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.718775 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.718777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.719128 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.719130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.719193 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.719211 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:24:05.719377 (+ 166us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:24:05.719379 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:24:05.719431 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:24:05.719480 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
I20260619 08:24:05.721943 1447 raft_consensus.cc:1261] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b [term 1 FOLLOWER]: Refusing update from remote peer 16e493ae050f42d38bf36e60e629b78e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:24:05.722188 1669 consensus_queue.cc:1035] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:24:05.722292 1307 raft_consensus.cc:1261] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662 [term 1 FOLLOWER]: Refusing update from remote peer 16e493ae050f42d38bf36e60e629b78e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:24:05.722515 1669 consensus_queue.cc:1035] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:24:05.730557 1657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.727935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:41661 (local address 127.25.127.131:45551)
0619 08:24:05.727997 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.727999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.728023 (+ 24us) server_negotiation.cc:407] Connection header received
0619 08:24:05.728123 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.728124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.728134 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.728192 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:24:05.728377 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.728849 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.729750 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.730054 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.730117 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.730348 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:24:05.730374 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:24:05.730413 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.730467 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.730483 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 08:24:05.730628 1718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.727732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:45551 (local address 127.25.127.129:41661)
0619 08:24:05.727923 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:05.727935 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:05.728065 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:05.728229 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:05.728232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:05.728233 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:24:05.728293 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.728295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.728931 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.728932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.729598 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:05.729600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.730167 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.730169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:05.730244 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.730263 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:24:05.730414 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:24:05.730416 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:24:05.730464 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 08:24:05.730509 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
I20260619 08:24:05.740229 1148 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:24:05.740341 1148 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:24:05.740967 1148 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932: Sending DeleteTablet for 3 replicas of tablet 332197f981744fba96d26fe60c7bbd65
I20260619 08:24:05.741254 1287 tablet_service.cc:1514] Processing DeleteTablet for tablet 332197f981744fba96d26fe60c7bbd65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:05 UTC) from {username='oryx', principal='oryx/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:42498
I20260619 08:24:05.741251 1427 tablet_service.cc:1514] Processing DeleteTablet for tablet 332197f981744fba96d26fe60c7bbd65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:05 UTC) from {username='oryx', principal='oryx/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:38808
I20260619 08:24:05.741357 1684 tablet_replica.cc:330] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b: stopping tablet replica
I20260619 08:24:05.741358 1683 tablet_replica.cc:330] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662: stopping tablet replica
I20260619 08:24:05.741410 1684 raft_consensus.cc:2229] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:24:05.741412 1683 raft_consensus.cc:2229] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:24:05.741473 1683 raft_consensus.cc:2258] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:24:05.741479 1684 raft_consensus.cc:2258] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:24:05.741770 1684 ts_tablet_manager.cc:1905] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:05.741804 1683 ts_tablet_manager.cc:1905] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:05.743418 1684 ts_tablet_manager.cc:1918] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:24:05.743486 1684 log.cc:1196] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/332197f981744fba96d26fe60c7bbd65
I20260619 08:24:05.743515 1567 tablet_service.cc:1514] Processing DeleteTablet for tablet 332197f981744fba96d26fe60c7bbd65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:05 UTC) from {username='oryx', principal='oryx/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:55170
I20260619 08:24:05.743606 1685 tablet_replica.cc:330] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e: stopping tablet replica
I20260619 08:24:05.743652 1685 raft_consensus.cc:2229] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:24:05.743705 1684 ts_tablet_manager.cc:1939] T 332197f981744fba96d26fe60c7bbd65 P a818706af2c4422b8dc86b89f502272b: Deleting consensus metadata
I20260619 08:24:05.743760 1685 raft_consensus.cc:2258] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:24:05.743961 1683 ts_tablet_manager.cc:1918] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:24:05.744031 1683 log.cc:1196] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/wal/wals/332197f981744fba96d26fe60c7bbd65
I20260619 08:24:05.744287 1683 ts_tablet_manager.cc:1939] T 332197f981744fba96d26fe60c7bbd65 P a39e357cd3814cd69a9a5b91fb41b662: Deleting consensus metadata
I20260619 08:24:05.746572 1685 ts_tablet_manager.cc:1905] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:05.748188 1685 ts_tablet_manager.cc:1918] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:24:05.748250 1685 log.cc:1196] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/332197f981744fba96d26fe60c7bbd65
I20260619 08:24:05.748451 1685 ts_tablet_manager.cc:1939] T 332197f981744fba96d26fe60c7bbd65 P 16e493ae050f42d38bf36e60e629b78e: Deleting consensus metadata
I20260619 08:24:05.748790 1136 catalog_manager.cc:4841] TS 16e493ae050f42d38bf36e60e629b78e (127.25.127.131:45551): tablet 332197f981744fba96d26fe60c7bbd65 (table test-table [id=9808776685054a5fb13c475914087959]) successfully deleted
I20260619 08:24:05.749172 1134 catalog_manager.cc:4841] TS a39e357cd3814cd69a9a5b91fb41b662 (127.25.127.129:43789): tablet 332197f981744fba96d26fe60c7bbd65 (table test-table [id=9808776685054a5fb13c475914087959]) successfully deleted
I20260619 08:24:05.749392 1134 catalog_manager.cc:4841] TS a818706af2c4422b8dc86b89f502272b (127.25.127.130:40921): tablet 332197f981744fba96d26fe60c7bbd65 (table test-table [id=9808776685054a5fb13c475914087959]) successfully deleted
Jun 19 08:24:05 dist-test-slave-5rhg krb5kdc[1105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 08:24:05 dist-test-slave-5rhg krb5kdc[1105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:05 dist-test-slave-5rhg krb5kdc[1105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 08:24:05.783221 1212 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:05.774615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49272 (local address 127.25.127.190:38215)
0619 08:24:05.774664 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.774667 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.776389 (+ 1722us) server_negotiation.cc:407] Connection header received
0619 08:24:05.776457 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.776458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.776473 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.776545 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:05.777305 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.778080 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.779197 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.779480 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.779550 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.783148 (+ 3598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49272: BlockingRecv error: recv got EOF from 127.0.0.1:49272 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 08:24:05 dist-test-slave-5rhg krb5kdc[1105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM
I20260619 08:24:05.797305 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.784506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49280 (local address 127.25.127.190:38215)
0619 08:24:05.784556 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.784558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.788365 (+ 3807us) server_negotiation.cc:407] Connection header received
0619 08:24:05.788422 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.788423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.788436 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.788504 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:05.789130 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.789901 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.790926 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.791240 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.791312 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.794185 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:05.794186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:05.794187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:05.794195 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:05.795955 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:05.796381 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:05.796383 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:05.796383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:05.796450 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:05.796681 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:05.796683 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:05.796683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:05.796830 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:05.796934 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.797118 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.797204 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:24:05.799439 1148 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:24:05.799623 1148 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:24:05.801237 1567 tablet_service.cc:1467] Processing CreateTablet for tablet 7cc10d1c64504494be711c13eb05b10a (DEFAULT_TABLE table=test-table [id=970ce40f6c714052a41a7c21fd96d4a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:24:05.801278 1287 tablet_service.cc:1467] Processing CreateTablet for tablet 7cc10d1c64504494be711c13eb05b10a (DEFAULT_TABLE table=test-table [id=970ce40f6c714052a41a7c21fd96d4a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:24:05.801393 1567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cc10d1c64504494be711c13eb05b10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:05.801406 1287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cc10d1c64504494be711c13eb05b10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:05.802711 1661 tablet_bootstrap.cc:492] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662: Bootstrap starting.
I20260619 08:24:05.802855 1427 tablet_service.cc:1467] Processing CreateTablet for tablet 7cc10d1c64504494be711c13eb05b10a (DEFAULT_TABLE table=test-table [id=970ce40f6c714052a41a7c21fd96d4a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:24:05.802981 1427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cc10d1c64504494be711c13eb05b10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:05.803128 1661 tablet_bootstrap.cc:654] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:05.804395 1662 tablet_bootstrap.cc:492] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b: Bootstrap starting.
I20260619 08:24:05.804410 1663 tablet_bootstrap.cc:492] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e: Bootstrap starting.
I20260619 08:24:05.804697 1662 tablet_bootstrap.cc:654] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:05.804807 1663 tablet_bootstrap.cc:654] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:05.805344 1662 tablet_bootstrap.cc:492] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b: No bootstrap required, opened a new log
I20260619 08:24:05.805383 1662 ts_tablet_manager.cc:1397] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 08:24:05.805509 1662 raft_consensus.cc:348] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.805599 1662 raft_consensus.cc:374] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:05.805619 1662 raft_consensus.cc:729] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a818706af2c4422b8dc86b89f502272b, State: Initialized, Role: FOLLOWER
I20260619 08:24:05.805672 1662 consensus_queue.cc:260] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.805758 1662 ts_tablet_manager.cc:1428] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:05.808522 1661 tablet_bootstrap.cc:492] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662: No bootstrap required, opened a new log
I20260619 08:24:05.808585 1661 ts_tablet_manager.cc:1397] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260619 08:24:05.808753 1661 raft_consensus.cc:348] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.808830 1661 raft_consensus.cc:374] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:05.808851 1661 raft_consensus.cc:729] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a39e357cd3814cd69a9a5b91fb41b662, State: Initialized, Role: FOLLOWER
I20260619 08:24:05.808892 1661 consensus_queue.cc:260] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.808976 1661 ts_tablet_manager.cc:1428] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:24:05.809334 1663 tablet_bootstrap.cc:492] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e: No bootstrap required, opened a new log
I20260619 08:24:05.809383 1663 ts_tablet_manager.cc:1397] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 08:24:05.809516 1663 raft_consensus.cc:348] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.809574 1663 raft_consensus.cc:374] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:05.809592 1663 raft_consensus.cc:729] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16e493ae050f42d38bf36e60e629b78e, State: Initialized, Role: FOLLOWER
I20260619 08:24:05.809630 1663 consensus_queue.cc:260] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.809702 1663 ts_tablet_manager.cc:1428] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:05.842271 1671 raft_consensus.cc:482] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:24:05.842391 1671 raft_consensus.cc:504] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.842557 1671 leader_election.cc:288] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a39e357cd3814cd69a9a5b91fb41b662 (127.25.127.129:43789), 16e493ae050f42d38bf36e60e629b78e (127.25.127.131:45551)
I20260619 08:24:05.842800 1587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cc10d1c64504494be711c13eb05b10a" candidate_uuid: "a818706af2c4422b8dc86b89f502272b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16e493ae050f42d38bf36e60e629b78e" is_pre_election: true
I20260619 08:24:05.842809 1307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cc10d1c64504494be711c13eb05b10a" candidate_uuid: "a818706af2c4422b8dc86b89f502272b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a39e357cd3814cd69a9a5b91fb41b662" is_pre_election: true
I20260619 08:24:05.842902 1587 raft_consensus.cc:2454] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a818706af2c4422b8dc86b89f502272b in term 0.
I20260619 08:24:05.842918 1307 raft_consensus.cc:2454] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a818706af2c4422b8dc86b89f502272b in term 0.
I20260619 08:24:05.843084 1380 leader_election.cc:302] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16e493ae050f42d38bf36e60e629b78e, a818706af2c4422b8dc86b89f502272b; no voters:
I20260619 08:24:05.843190 1671 raft_consensus.cc:2781] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:24:05.843219 1671 raft_consensus.cc:482] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:24:05.843238 1671 raft_consensus.cc:3037] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:05.844247 1671 raft_consensus.cc:504] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.844399 1671 leader_election.cc:288] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [CANDIDATE]: Term 1 election: Requested vote from peers a39e357cd3814cd69a9a5b91fb41b662 (127.25.127.129:43789), 16e493ae050f42d38bf36e60e629b78e (127.25.127.131:45551)
I20260619 08:24:05.844591 1307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cc10d1c64504494be711c13eb05b10a" candidate_uuid: "a818706af2c4422b8dc86b89f502272b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a39e357cd3814cd69a9a5b91fb41b662"
I20260619 08:24:05.844666 1307 raft_consensus.cc:3037] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:05.844638 1587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cc10d1c64504494be711c13eb05b10a" candidate_uuid: "a818706af2c4422b8dc86b89f502272b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16e493ae050f42d38bf36e60e629b78e"
I20260619 08:24:05.844714 1587 raft_consensus.cc:3037] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:05.845499 1307 raft_consensus.cc:2454] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a818706af2c4422b8dc86b89f502272b in term 1.
I20260619 08:24:05.845530 1587 raft_consensus.cc:2454] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a818706af2c4422b8dc86b89f502272b in term 1.
I20260619 08:24:05.845691 1378 leader_election.cc:302] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a39e357cd3814cd69a9a5b91fb41b662, a818706af2c4422b8dc86b89f502272b; no voters:
I20260619 08:24:05.845800 1671 raft_consensus.cc:2781] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:24:05.845912 1671 raft_consensus.cc:686] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [term 1 LEADER]: Becoming Leader. State: Replica: a818706af2c4422b8dc86b89f502272b, State: Running, Role: LEADER
I20260619 08:24:05.845974 1671 consensus_queue.cc:237] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.846551 1148 catalog_manager.cc:5495] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b reported cstate change: term changed from 0 to 1, leader changed from <none> to a818706af2c4422b8dc86b89f502272b (127.25.127.130). New cstate: current_term: 1 leader_uuid: "a818706af2c4422b8dc86b89f502272b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } health_report { overall_health: UNKNOWN } } }
I20260619 08:24:05.858749 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:05.855428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38826 (local address 127.25.127.130:40921)
0619 08:24:05.855478 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:05.855480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:05.855491 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:24:05.855544 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:05.855545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:05.855556 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:05.855620 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:24:05.855762 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.856512 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.857884 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:05.858188 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:05.858268 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:05.858461 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:24:05.858502 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:24:05.858563 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:05.858637 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 08:24:05.858665 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:24:05.859643 1307 raft_consensus.cc:1261] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662 [term 1 FOLLOWER]: Refusing update from remote peer a818706af2c4422b8dc86b89f502272b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:24:05.859664 1587 raft_consensus.cc:1261] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e [term 1 FOLLOWER]: Refusing update from remote peer a818706af2c4422b8dc86b89f502272b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:24:05.859879 1671 consensus_queue.cc:1035] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [LEADER]: Connected to new peer: Peer: permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:24:05.860002 1736 consensus_queue.cc:1035] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:24:05.863998 1148 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:24:05.864099 1148 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:24:05.864766 1148 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932: Sending DeleteTablet for 3 replicas of tablet 7cc10d1c64504494be711c13eb05b10a
I20260619 08:24:05.864990 1287 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cc10d1c64504494be711c13eb05b10a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:05 UTC) from {username='oryx', principal='oryx/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:42498
I20260619 08:24:05.865005 1427 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cc10d1c64504494be711c13eb05b10a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:05 UTC) from {username='oryx', principal='oryx/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:38808
I20260619 08:24:05.865087 1684 tablet_replica.cc:330] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b: stopping tablet replica
I20260619 08:24:05.865088 1683 tablet_replica.cc:330] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662: stopping tablet replica
I20260619 08:24:05.865130 1684 raft_consensus.cc:2229] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:24:05.865131 1683 raft_consensus.cc:2229] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:24:05.865178 1684 raft_consensus.cc:2258] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:24:05.865180 1683 raft_consensus.cc:2258] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:24:05.865474 1567 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cc10d1c64504494be711c13eb05b10a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:05 UTC) from {username='oryx', principal='oryx/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:55170
I20260619 08:24:05.865553 1685 tablet_replica.cc:330] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e: stopping tablet replica
I20260619 08:24:05.865599 1685 raft_consensus.cc:2229] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:24:05.865593 1684 ts_tablet_manager.cc:1905] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:05.865638 1685 raft_consensus.cc:2258] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:24:05.866194 1148 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:24:05.866336 1148 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:24:05.867007 1684 ts_tablet_manager.cc:1918] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:24:05.867061 1684 log.cc:1196] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/7cc10d1c64504494be711c13eb05b10a
I20260619 08:24:05.867285 1684 ts_tablet_manager.cc:1939] T 7cc10d1c64504494be711c13eb05b10a P a818706af2c4422b8dc86b89f502272b: Deleting consensus metadata
I20260619 08:24:05.867595 1134 catalog_manager.cc:4841] TS a818706af2c4422b8dc86b89f502272b (127.25.127.130:40921): tablet 7cc10d1c64504494be711c13eb05b10a (table test-table [id=970ce40f6c714052a41a7c21fd96d4a7]) successfully deleted
I20260619 08:24:05.867908 1567 tablet_service.cc:1467] Processing CreateTablet for tablet 8fb74a3b5e1d4fd6950d2e6229f1a181 (DEFAULT_TABLE table=test-table [id=614f2cdffa344c53a26663504e876198]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:24:05.867911 1287 tablet_service.cc:1467] Processing CreateTablet for tablet 8fb74a3b5e1d4fd6950d2e6229f1a181 (DEFAULT_TABLE table=test-table [id=614f2cdffa344c53a26663504e876198]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:24:05.868019 1287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fb74a3b5e1d4fd6950d2e6229f1a181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:05.868016 1567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fb74a3b5e1d4fd6950d2e6229f1a181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:05.868264 1427 tablet_service.cc:1467] Processing CreateTablet for tablet 8fb74a3b5e1d4fd6950d2e6229f1a181 (DEFAULT_TABLE table=test-table [id=614f2cdffa344c53a26663504e876198]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:24:05.868346 1427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fb74a3b5e1d4fd6950d2e6229f1a181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:05.869445 1663 tablet_bootstrap.cc:492] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e: Bootstrap starting.
I20260619 08:24:05.869892 1663 tablet_bootstrap.cc:654] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:05.870121 1683 ts_tablet_manager.cc:1905] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:05.870234 1662 tablet_bootstrap.cc:492] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b: Bootstrap starting.
I20260619 08:24:05.870523 1662 tablet_bootstrap.cc:654] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:05.871093 1662 tablet_bootstrap.cc:492] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b: No bootstrap required, opened a new log
I20260619 08:24:05.871130 1662 ts_tablet_manager.cc:1397] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 08:24:05.871228 1662 raft_consensus.cc:348] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.871290 1662 raft_consensus.cc:374] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:05.871308 1662 raft_consensus.cc:729] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a818706af2c4422b8dc86b89f502272b, State: Initialized, Role: FOLLOWER
I20260619 08:24:05.871346 1662 consensus_queue.cc:260] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.871436 1662 ts_tablet_manager.cc:1428] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:05.871548 1683 ts_tablet_manager.cc:1918] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:24:05.871565 1661 tablet_bootstrap.cc:492] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a39e357cd3814cd69a9a5b91fb41b662: Bootstrap starting.
I20260619 08:24:05.871604 1683 log.cc:1196] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-0/wal/wals/7cc10d1c64504494be711c13eb05b10a
I20260619 08:24:05.871871 1683 ts_tablet_manager.cc:1939] T 7cc10d1c64504494be711c13eb05b10a P a39e357cd3814cd69a9a5b91fb41b662: Deleting consensus metadata
I20260619 08:24:05.871874 1661 tablet_bootstrap.cc:654] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a39e357cd3814cd69a9a5b91fb41b662: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:05.872203 1663 tablet_bootstrap.cc:492] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e: No bootstrap required, opened a new log
I20260619 08:24:05.872246 1663 ts_tablet_manager.cc:1397] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 08:24:05.872366 1685 ts_tablet_manager.cc:1905] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:05.872392 1663 raft_consensus.cc:348] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.872452 1663 raft_consensus.cc:374] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:05.872470 1663 raft_consensus.cc:729] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16e493ae050f42d38bf36e60e629b78e, State: Initialized, Role: FOLLOWER
I20260619 08:24:05.872514 1663 consensus_queue.cc:260] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.872592 1663 ts_tablet_manager.cc:1428] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:05.872707 1134 catalog_manager.cc:4841] TS a39e357cd3814cd69a9a5b91fb41b662 (127.25.127.129:43789): tablet 7cc10d1c64504494be711c13eb05b10a (table test-table [id=970ce40f6c714052a41a7c21fd96d4a7]) successfully deleted
I20260619 08:24:05.873447 1661 tablet_bootstrap.cc:492] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a39e357cd3814cd69a9a5b91fb41b662: No bootstrap required, opened a new log
I20260619 08:24:05.873488 1661 ts_tablet_manager.cc:1397] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a39e357cd3814cd69a9a5b91fb41b662: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260619 08:24:05.873579 1661 raft_consensus.cc:348] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.873638 1661 raft_consensus.cc:374] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:05.873656 1661 raft_consensus.cc:729] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a39e357cd3814cd69a9a5b91fb41b662, State: Initialized, Role: FOLLOWER
I20260619 08:24:05.873698 1661 consensus_queue.cc:260] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a39e357cd3814cd69a9a5b91fb41b662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:05.873772 1661 ts_tablet_manager.cc:1428] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a39e357cd3814cd69a9a5b91fb41b662: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:05.873804 1685 ts_tablet_manager.cc:1918] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:24:05.873853 1685 log.cc:1196] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/7cc10d1c64504494be711c13eb05b10a
I20260619 08:24:05.874037 1685 ts_tablet_manager.cc:1939] T 7cc10d1c64504494be711c13eb05b10a P 16e493ae050f42d38bf36e60e629b78e: Deleting consensus metadata
I20260619 08:24:05.874502 1136 catalog_manager.cc:4841] TS 16e493ae050f42d38bf36e60e629b78e (127.25.127.131:45551): tablet 7cc10d1c64504494be711c13eb05b10a (table test-table [id=970ce40f6c714052a41a7c21fd96d4a7]) successfully deleted
I20260619 08:24:06.040654 1671 raft_consensus.cc:482] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:24:06.040781 1671 raft_consensus.cc:504] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:06.041013 1671 leader_election.cc:288] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a39e357cd3814cd69a9a5b91fb41b662 (127.25.127.129:43789), 16e493ae050f42d38bf36e60e629b78e (127.25.127.131:45551)
I20260619 08:24:06.041249 1307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb74a3b5e1d4fd6950d2e6229f1a181" candidate_uuid: "a818706af2c4422b8dc86b89f502272b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a39e357cd3814cd69a9a5b91fb41b662" is_pre_election: true
I20260619 08:24:06.041366 1307 raft_consensus.cc:2454] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a818706af2c4422b8dc86b89f502272b in term 0.
I20260619 08:24:06.041301 1587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb74a3b5e1d4fd6950d2e6229f1a181" candidate_uuid: "a818706af2c4422b8dc86b89f502272b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16e493ae050f42d38bf36e60e629b78e" is_pre_election: true
I20260619 08:24:06.041425 1587 raft_consensus.cc:2454] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a818706af2c4422b8dc86b89f502272b in term 0.
I20260619 08:24:06.041548 1378 leader_election.cc:302] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a39e357cd3814cd69a9a5b91fb41b662, a818706af2c4422b8dc86b89f502272b; no voters:
I20260619 08:24:06.041674 1671 raft_consensus.cc:2781] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:24:06.041751 1671 raft_consensus.cc:482] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:24:06.041785 1671 raft_consensus.cc:3037] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:06.042523 1671 raft_consensus.cc:504] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:06.042683 1671 leader_election.cc:288] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [CANDIDATE]: Term 1 election: Requested vote from peers a39e357cd3814cd69a9a5b91fb41b662 (127.25.127.129:43789), 16e493ae050f42d38bf36e60e629b78e (127.25.127.131:45551)
I20260619 08:24:06.042863 1587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb74a3b5e1d4fd6950d2e6229f1a181" candidate_uuid: "a818706af2c4422b8dc86b89f502272b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16e493ae050f42d38bf36e60e629b78e"
I20260619 08:24:06.042847 1307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb74a3b5e1d4fd6950d2e6229f1a181" candidate_uuid: "a818706af2c4422b8dc86b89f502272b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a39e357cd3814cd69a9a5b91fb41b662"
I20260619 08:24:06.042932 1587 raft_consensus.cc:3037] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:06.042958 1307 raft_consensus.cc:3037] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a39e357cd3814cd69a9a5b91fb41b662 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:06.043592 1587 raft_consensus.cc:2454] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a818706af2c4422b8dc86b89f502272b in term 1.
I20260619 08:24:06.043606 1307 raft_consensus.cc:2454] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a39e357cd3814cd69a9a5b91fb41b662 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a818706af2c4422b8dc86b89f502272b in term 1.
I20260619 08:24:06.043766 1380 leader_election.cc:302] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16e493ae050f42d38bf36e60e629b78e, a818706af2c4422b8dc86b89f502272b; no voters:
I20260619 08:24:06.043879 1671 raft_consensus.cc:2781] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:24:06.043972 1671 raft_consensus.cc:686] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [term 1 LEADER]: Becoming Leader. State: Replica: a818706af2c4422b8dc86b89f502272b, State: Running, Role: LEADER
I20260619 08:24:06.044032 1671 consensus_queue.cc:237] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } }
I20260619 08:24:06.044698 1148 catalog_manager.cc:5495] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b reported cstate change: term changed from 0 to 1, leader changed from <none> to a818706af2c4422b8dc86b89f502272b (127.25.127.130). New cstate: current_term: 1 leader_uuid: "a818706af2c4422b8dc86b89f502272b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a818706af2c4422b8dc86b89f502272b" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 40921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 } health_report { overall_health: UNKNOWN } } }
I20260619 08:24:06.063764 1660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:06.061326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:36499 (local address 127.25.127.129:43789)
0619 08:24:06.061426 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:06.061429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:06.061497 (+ 68us) server_negotiation.cc:407] Connection header received
0619 08:24:06.061577 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:06.061578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:06.061589 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:06.061639 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:24:06.061749 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:06.062234 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:06.063008 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:06.063224 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:06.063300 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:06.063490 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:24:06.063527 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:24:06.063595 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:06.063658 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 08:24:06.063682 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 08:24:06.063791 1747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:06.061248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:43789 (local address 127.25.127.130:36499)
0619 08:24:06.061442 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:06.061453 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:06.061515 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:06.061641 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:06.061642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:06.061644 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:24:06.061699 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:06.061701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:06.062321 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:06.062323 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:06.062921 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:06.062923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:06.063304 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:06.063305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:06.063373 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:06.063394 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:24:06.063598 (+ 204us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:24:06.063600 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:24:06.063654 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:24:06.063709 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":106,"threads_started":1}
I20260619 08:24:06.066135 1587 raft_consensus.cc:1261] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e [term 1 FOLLOWER]: Refusing update from remote peer a818706af2c4422b8dc86b89f502272b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:24:06.066286 1307 raft_consensus.cc:1261] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a39e357cd3814cd69a9a5b91fb41b662 [term 1 FOLLOWER]: Refusing update from remote peer a818706af2c4422b8dc86b89f502272b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:24:06.066340 1736 consensus_queue.cc:1035] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [LEADER]: Connected to new peer: Peer: permanent_uuid: "16e493ae050f42d38bf36e60e629b78e" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:24:06.066500 1671 consensus_queue.cc:1035] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39e357cd3814cd69a9a5b91fb41b662" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:24:06.073426 1718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:06.071041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:40921 (local address 127.25.127.129:37661)
0619 08:24:06.071075 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:06.071086 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:06.071122 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:06.071351 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:06.071351 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:06.071352 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:24:06.071387 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:06.071389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:06.072044 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:06.072045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:06.072656 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:06.072657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:06.073031 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:06.073032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:06.073086 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:06.073093 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:24:06.073273 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:24:06.073274 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:24:06.073314 (+ 40us) client_negotiation.cc:240] Negotiation successful
0619 08:24:06.073358 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260619 08:24:06.073421 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:06.071217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:37661 (local address 127.25.127.130:40921)
0619 08:24:06.071257 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:06.071258 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:06.071265 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:24:06.071282 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:06.071282 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:06.071290 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:06.071328 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:24:06.071464 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:06.071950 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:06.072738 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:06.072949 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:06.073031 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:06.073182 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:24:06.073215 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:24:06.073265 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:06.073325 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 08:24:06.073345 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:24:06.079640 1149 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:24:06.079757 1149 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:24:06.080461 1149 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5695ede7410c499e9173818bbae01932: Sending DeleteTablet for 3 replicas of tablet 8fb74a3b5e1d4fd6950d2e6229f1a181
I20260619 08:24:06.080711 1567 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fb74a3b5e1d4fd6950d2e6229f1a181 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:06 UTC) from {username='oryx', principal='oryx/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:55170
I20260619 08:24:06.080749 1427 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fb74a3b5e1d4fd6950d2e6229f1a181 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:06 UTC) from {username='oryx', principal='oryx/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:38808
I20260619 08:24:06.080828 1685 tablet_replica.cc:330] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e: stopping tablet replica
I20260619 08:24:06.080859 1684 tablet_replica.cc:330] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b: stopping tablet replica
I20260619 08:24:06.080881 1685 raft_consensus.cc:2229] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:24:06.080924 1685 raft_consensus.cc:2258] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:24:06.080916 1684 raft_consensus.cc:2229] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:24:06.080904 1287 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fb74a3b5e1d4fd6950d2e6229f1a181 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:06 UTC) from {username='oryx', principal='oryx/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:42498
I20260619 08:24:06.080982 1684 raft_consensus.cc:2258] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:24:06.081060 1683 tablet_replica.cc:330] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a39e357cd3814cd69a9a5b91fb41b662: stopping tablet replica
I20260619 08:24:06.081122 1683 raft_consensus.cc:2229] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a39e357cd3814cd69a9a5b91fb41b662 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:24:06.081168 1683 raft_consensus.cc:2258] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a39e357cd3814cd69a9a5b91fb41b662 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:24:06.081321 1685 ts_tablet_manager.cc:1905] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:06.081470 1684 ts_tablet_manager.cc:1905] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:06.081493 1683 ts_tablet_manager.cc:1905] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a39e357cd3814cd69a9a5b91fb41b662: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:06.081509 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 1222
I20260619 08:24:06.082969 1685 ts_tablet_manager.cc:1918] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:24:06.082988 1684 ts_tablet_manager.cc:1918] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:24:06.083034 1684 log.cc:1196] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/8fb74a3b5e1d4fd6950d2e6229f1a181
I20260619 08:24:06.083034 1685 log.cc:1196] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/8fb74a3b5e1d4fd6950d2e6229f1a181
I20260619 08:24:06.083250 1684 ts_tablet_manager.cc:1939] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P a818706af2c4422b8dc86b89f502272b: Deleting consensus metadata
I20260619 08:24:06.083250 1685 ts_tablet_manager.cc:1939] T 8fb74a3b5e1d4fd6950d2e6229f1a181 P 16e493ae050f42d38bf36e60e629b78e: Deleting consensus metadata
I20260619 08:24:06.083731 1134 catalog_manager.cc:4841] TS a818706af2c4422b8dc86b89f502272b (127.25.127.130:40921): tablet 8fb74a3b5e1d4fd6950d2e6229f1a181 (table test-table [id=614f2cdffa344c53a26663504e876198]) successfully deleted
I20260619 08:24:06.083762 1136 catalog_manager.cc:4841] TS 16e493ae050f42d38bf36e60e629b78e (127.25.127.131:45551): tablet 8fb74a3b5e1d4fd6950d2e6229f1a181 (table test-table [id=614f2cdffa344c53a26663504e876198]) successfully deleted
W20260619 08:24:06.089720 1134 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.127.129:43789 (error 108)
I20260619 08:24:06.090085 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 1362
I20260619 08:24:06.091235 1752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:06.090181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:43789 (local address 127.0.0.1:42502)
0619 08:24:06.091121 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:06.091179 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.129:43789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":921,"thread_start_us":46,"threads_started":1}
W20260619 08:24:06.091341 1134 catalog_manager.cc:4584] TS a39e357cd3814cd69a9a5b91fb41b662 (127.25.127.129:43789): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8fb74a3b5e1d4fd6950d2e6229f1a181: Network error: Client connection negotiation failed: client connection to 127.25.127.129:43789: connect: Connection refused (error 111)
I20260619 08:24:06.095973 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 1502
I20260619 08:24:06.102257 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 1118
2026-06-19T08:24:06Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2718 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:24:06 dist-test-slave-5rhg krb5kdc[1755](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:24:06 dist-test-slave-5rhg krb5kdc[1755](info): set up 2 sockets
Jun 19 08:24:06 dist-test-slave-5rhg krb5kdc[1755](info): commencing operation
krb5kdc: starting...
W20260619 08:24:07.149717 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:24:07 dist-test-slave-5rhg krb5kdc[1755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:24:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:24:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:24:07.269073 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:35941
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:39789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:35941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:24:07.354490 1768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:07.354712 1768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:07.354736 1768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:07.355862 1768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:24:07.355901 1768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:07.355914 1768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:07.355927 1768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:24:07.355939 1768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:24:07.357373 1768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:39789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:35941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:35941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:07.357573 1768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:07.357748 1768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:07.362737 1773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:07.362772 1768 server_base.cc:1034] running on GCE node
W20260619 08:24:07.362715 1776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:07.362701 1774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:07.363169 1768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:07.363432 1768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:07.364570 1768 hybrid_clock.cc:648] HybridClock initialized: now 1781857447364558 us; error 29 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T08:24:07Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1263 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:24:07 dist-test-slave-5rhg krb5kdc[1781](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:24:07 dist-test-slave-5rhg krb5kdc[1781](info): set up 2 sockets
Jun 19 08:24:07 dist-test-slave-5rhg krb5kdc[1781](info): commencing operation
krb5kdc: starting...
W20260619 08:24:08.405611 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:24:08 dist-test-slave-5rhg krb5kdc[1781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:24:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:24:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:24:08.527974 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:36493
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:39963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:36493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:24:08.613612 1794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:08.613840 1794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:08.613870 1794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:08.615116 1794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:24:08.615162 1794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:08.615177 1794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:08.615227 1794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:24:08.615255 1794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:24:08.616843 1794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:39963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:36493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:36493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:08.617123 1794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:08.617365 1794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:08.622282 1802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:08.622254 1799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:08.622267 1800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:08.622610 1794 server_base.cc:1034] running on GCE node
I20260619 08:24:08.622908 1794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:08.623196 1794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:08.624336 1794 hybrid_clock.cc:648] HybridClock initialized: now 1781857448624324 us; error 33 us; skew 500 ppm
Jun 19 08:24:08 dist-test-slave-5rhg krb5kdc[1781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857448, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:24:08.626987 1794 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:24:08.627489 1794 webserver.cc:460] Webserver started at http://127.25.127.190:34451/ using document root <none> and password file <none>
I20260619 08:24:08.627736 1794 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:08.627805 1794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:08.627943 1794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:08.629092 1794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "401091590ca84968a229cad6397e0f43"
format_stamp: "Formatted at 2026-06-19 08:24:08 on dist-test-slave-5rhg"
server_key: "dfe8cf7aa5a09647eb761cd9f7595a5f"
server_key_iv: "dacaee908914bc2463c4b3d0314a7b90"
server_key_version: "encryptionkey@0"
I20260619 08:24:08.629474 1794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "401091590ca84968a229cad6397e0f43"
format_stamp: "Formatted at 2026-06-19 08:24:08 on dist-test-slave-5rhg"
server_key: "dfe8cf7aa5a09647eb761cd9f7595a5f"
server_key_iv: "dacaee908914bc2463c4b3d0314a7b90"
server_key_version: "encryptionkey@0"
I20260619 08:24:08.630860 1794 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:24:08.631702 1809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:08.631901 1794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:08.632010 1794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "401091590ca84968a229cad6397e0f43"
format_stamp: "Formatted at 2026-06-19 08:24:08 on dist-test-slave-5rhg"
server_key: "dfe8cf7aa5a09647eb761cd9f7595a5f"
server_key_iv: "dacaee908914bc2463c4b3d0314a7b90"
server_key_version: "encryptionkey@0"
I20260619 08:24:08.632093 1794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:08.646889 1794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:08.649009 1794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:08.649183 1794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:08.653436 1794 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:36493
I20260619 08:24:08.653486 1861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:36493 every 8 connection(s)
I20260619 08:24:08.653867 1794 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:24:08.654716 1862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:08.657064 1862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43: Bootstrap starting.
I20260619 08:24:08.657603 1862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:08.657866 1862 log.cc:824] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43: Log is configured to *not* fsync() on all Append() calls
I20260619 08:24:08.658563 1862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43: No bootstrap required, opened a new log
I20260619 08:24:08.660200 1862 raft_consensus.cc:348] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "401091590ca84968a229cad6397e0f43" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36493 } }
I20260619 08:24:08.660362 1862 raft_consensus.cc:374] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:08.660384 1862 raft_consensus.cc:729] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 401091590ca84968a229cad6397e0f43, State: Initialized, Role: FOLLOWER
I20260619 08:24:08.660522 1862 consensus_queue.cc:260] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "401091590ca84968a229cad6397e0f43" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36493 } }
I20260619 08:24:08.660597 1862 raft_consensus.cc:388] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:24:08.660619 1862 raft_consensus.cc:482] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:24:08.660647 1862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:08.661150 1862 raft_consensus.cc:504] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "401091590ca84968a229cad6397e0f43" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36493 } }
I20260619 08:24:08.661232 1862 leader_election.cc:302] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 401091590ca84968a229cad6397e0f43; no voters:
I20260619 08:24:08.661367 1862 leader_election.cc:288] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:24:08.661461 1865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:24:08.661636 1862 sys_catalog.cc:564] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:24:08.661914 1865 raft_consensus.cc:686] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [term 1 LEADER]: Becoming Leader. State: Replica: 401091590ca84968a229cad6397e0f43, State: Running, Role: LEADER
I20260619 08:24:08.662045 1865 consensus_queue.cc:237] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "401091590ca84968a229cad6397e0f43" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36493 } }
I20260619 08:24:08.662597 1866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "401091590ca84968a229cad6397e0f43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "401091590ca84968a229cad6397e0f43" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36493 } } }
I20260619 08:24:08.662694 1866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [sys.catalog]: This master's current role is: LEADER
I20260619 08:24:08.663081 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 1794
I20260619 08:24:08.663086 1867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 401091590ca84968a229cad6397e0f43. Latest consensus state: current_term: 1 leader_uuid: "401091590ca84968a229cad6397e0f43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "401091590ca84968a229cad6397e0f43" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36493 } } }
I20260619 08:24:08.663153 1867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [sys.catalog]: This master's current role is: LEADER
I20260619 08:24:08.663185 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:24:08.663398 26110 external_mini_cluster.cc:1428] Setting key f5c2e5508f8abc6dc15c36f3dd737075
W20260619 08:24:08.663604 1882 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:24:08.663703 1882 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:24:08.663817 1876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:24:08.664000 1876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:24:08.665791 1876 catalog_manager.cc:1347] Generated new cluster ID: 88fefa8b62074107bc06bc25d6763d74
I20260619 08:24:08.665844 1876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:24:08 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:24:08.675590 1885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:08.663964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41158 (local address 127.25.127.190:36493)
0619 08:24:08.664138 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:08.664172 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:08.664236 (+ 64us) server_negotiation.cc:407] Connection header received
0619 08:24:08.664604 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:08.664614 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:08.664684 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:08.664854 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:08.665641 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:08.667117 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:08.668203 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:08.668894 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:08.669013 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:08.671805 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:08.671810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:08.671810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:08.671823 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:08.673860 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:08.674286 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:08.674288 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:08.674289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:08.674400 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:08.674626 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:08.674626 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:08.674627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:08.674934 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:08.675074 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:08.675205 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 08:24:08.675344 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":70,"threads_started":1}
I20260619 08:24:08.690804 1876 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:24:08.691394 1876 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:24:08.696524 1876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43: Generated new TSK 0
I20260619 08:24:08.696741 1876 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
I20260619 08:24:08.731909 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:36493
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:39963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:24:08.842248 1889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:08.842478 1889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:08.842504 1889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:08.843746 1889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:08.843798 1889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:08.843827 1889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:24:08.845366 1889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:39963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:36493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:08.845643 1889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:08.845906 1889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:08.851133 1894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:08.851104 1897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:08.851114 1895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:08.851528 1889 server_base.cc:1034] running on GCE node
I20260619 08:24:08.851730 1889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:08.852032 1889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:08.853201 1889 hybrid_clock.cc:648] HybridClock initialized: now 1781857448853194 us; error 38 us; skew 500 ppm
Jun 19 08:24:08 dist-test-slave-5rhg krb5kdc[1781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857448, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:24:08.855752 1889 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:24:08.856269 1889 webserver.cc:460] Webserver started at http://127.25.127.129:37857/ using document root <none> and password file <none>
I20260619 08:24:08.856523 1889 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:08.856591 1889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:08.856739 1889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:08.857844 1889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "2908637f042d401f9ca9d6907379439b"
format_stamp: "Formatted at 2026-06-19 08:24:08 on dist-test-slave-5rhg"
server_key: "9284b2a414ae2e6236611ddc6a483a66"
server_key_iv: "0d7d68d84caae8e58d5a4eddc79ce23c"
server_key_version: "encryptionkey@0"
I20260619 08:24:08.858227 1889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "2908637f042d401f9ca9d6907379439b"
format_stamp: "Formatted at 2026-06-19 08:24:08 on dist-test-slave-5rhg"
server_key: "9284b2a414ae2e6236611ddc6a483a66"
server_key_iv: "0d7d68d84caae8e58d5a4eddc79ce23c"
server_key_version: "encryptionkey@0"
I20260619 08:24:08.859603 1889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:24:08.860476 1904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:08.860649 1889 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:24:08.860754 1889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "2908637f042d401f9ca9d6907379439b"
format_stamp: "Formatted at 2026-06-19 08:24:08 on dist-test-slave-5rhg"
server_key: "9284b2a414ae2e6236611ddc6a483a66"
server_key_iv: "0d7d68d84caae8e58d5a4eddc79ce23c"
server_key_version: "encryptionkey@0"
I20260619 08:24:08.860834 1889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:08.883656 1889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:08.885730 1889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:08.885906 1889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:08.886183 1889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:24:08.886518 1889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:24:08.886574 1889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:08.886610 1889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:24:08.886656 1889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:08.892544 1889 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:46825
I20260619 08:24:08.892590 2017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:46825 every 8 connection(s)
I20260619 08:24:08.892915 1889 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:24:08.898135 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 1889
I20260619 08:24:08.898226 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:24:08.898429 26110 external_mini_cluster.cc:1428] Setting key b8ae988e3e8404481c4b37f64062104c
Jun 19 08:24:08 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857448, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:24:08.902801 1885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:08.893600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:34341 (local address 127.25.127.190:36493)
0619 08:24:08.893676 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:08.893678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:08.893900 (+ 222us) server_negotiation.cc:407] Connection header received
0619 08:24:08.894413 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:08.894414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:08.894426 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:08.894502 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:08.895427 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:08.895932 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:08.896961 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:08.897166 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:08.897221 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:08.899626 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:08.899628 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:08.899628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:08.899637 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:08.901369 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:08.901812 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:08.901814 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:08.901814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:08.901871 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:08.902133 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:08.902134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:08.902135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:08.902254 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:08.902329 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:08.902619 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 08:24:08.902707 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 08:24:08.903020 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:08.893575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:34341)
0619 08:24:08.893753 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:08.893796 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:08.894319 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:08.894569 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:08.894573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:08.895146 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:08.895346 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:08.895350 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:08.896015 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:08.896016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:08.896889 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:08.896891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:08.897241 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:08.897241 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:08.897345 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:08.897849 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:08.897858 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:08.899540 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:08.901480 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:08.901488 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:08.901490 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:08.901721 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:08.901945 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:08.901946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:08.901946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:08.902064 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:08.902321 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:08.902327 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:08.902483 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 08:24:08.902597 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 08:24:08.902765 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":107,"threads_started":1}
I20260619 08:24:08.903617 2018 heartbeater.cc:347] Connected to a master server at 127.25.127.190:36493
I20260619 08:24:08.903730 2018 heartbeater.cc:464] Registering TS with master...
I20260619 08:24:08.903909 2018 heartbeater.cc:511] Master 127.25.127.190:36493 requested a full tablet report, sending...
I20260619 08:24:08.904500 1826 ts_manager.cc:194] Registered new tserver with Master: 2908637f042d401f9ca9d6907379439b (127.25.127.129:46825)
I20260619 08:24:08.905104 1826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:34341
WARNING: no policy specified for kudu/127.25.127.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
I20260619 08:24:08.923597 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:36493
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:39963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:24:09.006865 2023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:09.007071 2023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:09.007095 2023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:09.008304 2023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:09.008345 2023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:09.008371 2023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:24:09.009786 2023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:39963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:36493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:09.009984 2023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:09.010161 2023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:09.015489 2031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:09.015539 2023 server_base.cc:1034] running on GCE node
W20260619 08:24:09.015506 2029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:09.015506 2028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:09.015919 2023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:09.016243 2023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:09.017410 2023 hybrid_clock.cc:648] HybridClock initialized: now 1781857449017404 us; error 35 us; skew 500 ppm
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857449, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:24:09.019881 2023 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:24:09.020406 2023 webserver.cc:460] Webserver started at http://127.25.127.130:46103/ using document root <none> and password file <none>
I20260619 08:24:09.020658 2023 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:09.020728 2023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:09.020882 2023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:09.021922 2023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "656973288a8b4a06a92a9a77def54218"
format_stamp: "Formatted at 2026-06-19 08:24:09 on dist-test-slave-5rhg"
server_key: "8718b8661a0a56e4be1d6548466ff741"
server_key_iv: "b6c6a09b7d943ebec2ad1adcc8a2c766"
server_key_version: "encryptionkey@0"
I20260619 08:24:09.022285 2023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "656973288a8b4a06a92a9a77def54218"
format_stamp: "Formatted at 2026-06-19 08:24:09 on dist-test-slave-5rhg"
server_key: "8718b8661a0a56e4be1d6548466ff741"
server_key_iv: "b6c6a09b7d943ebec2ad1adcc8a2c766"
server_key_version: "encryptionkey@0"
I20260619 08:24:09.023581 2023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:24:09.024348 2038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:09.024525 2023 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:24:09.024627 2023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "656973288a8b4a06a92a9a77def54218"
format_stamp: "Formatted at 2026-06-19 08:24:09 on dist-test-slave-5rhg"
server_key: "8718b8661a0a56e4be1d6548466ff741"
server_key_iv: "b6c6a09b7d943ebec2ad1adcc8a2c766"
server_key_version: "encryptionkey@0"
I20260619 08:24:09.024701 2023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:09.033290 2023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:09.035403 2023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:09.035571 2023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:09.035802 2023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:24:09.036114 2023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:24:09.036194 2023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:09.036238 2023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:24:09.036269 2023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:09.042203 2023 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:34175
I20260619 08:24:09.042256 2151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:34175 every 8 connection(s)
I20260619 08:24:09.042569 2023 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:24:09.048697 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 2023
I20260619 08:24:09.048820 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:24:09.049073 26110 external_mini_cluster.cc:1428] Setting key ad32924c30207cce94374f626c45dd6b
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857449, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:24:09.053529 1885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:09.043291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:34707 (local address 127.25.127.190:36493)
0619 08:24:09.043391 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:09.043393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:09.043564 (+ 171us) server_negotiation.cc:407] Connection header received
0619 08:24:09.044073 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:09.044074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:09.044085 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:09.044183 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:09.045193 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.045739 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.046887 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.047135 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.047210 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.050015 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:09.050017 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:09.050017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:09.050025 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:09.051721 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:09.052289 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:09.052290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:09.052290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:09.052352 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:09.052695 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:09.052696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:09.052697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:09.052892 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:09.052960 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:09.053363 (+ 403us) server_negotiation.cc:299] Negotiation successful
0619 08:24:09.053456 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 08:24:09.053941 2154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:09.043247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:34707)
0619 08:24:09.043425 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:09.043476 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:09.043968 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:09.044282 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:09.044298 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:09.044877 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:09.045082 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.045087 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.045849 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.045850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.046784 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.046786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.047269 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.047270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.047459 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.048057 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:09.048066 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:09.049923 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:09.051808 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:09.051816 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:09.051819 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:09.052206 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:09.052428 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:09.052429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:09.052429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:09.052588 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:09.052976 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:09.052983 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:09.053177 (+ 194us) client_negotiation.cc:769] Sending connection context
0619 08:24:09.053297 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 08:24:09.053611 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":107,"threads_started":1}
I20260619 08:24:09.054540 2152 heartbeater.cc:347] Connected to a master server at 127.25.127.190:36493
I20260619 08:24:09.054667 2152 heartbeater.cc:464] Registering TS with master...
I20260619 08:24:09.054869 2152 heartbeater.cc:511] Master 127.25.127.190:36493 requested a full tablet report, sending...
I20260619 08:24:09.055325 1826 ts_manager.cc:194] Registered new tserver with Master: 656973288a8b4a06a92a9a77def54218 (127.25.127.130:34175)
I20260619 08:24:09.055720 1826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:34707
WARNING: no policy specified for kudu/127.25.127.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
I20260619 08:24:09.077071 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:36493
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:39963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:24:09.159147 2157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:09.159407 2157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:09.159451 2157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:09.160791 2157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:09.160868 2157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:09.160917 2157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:24:09.162750 2157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:39963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:36493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:09.163105 2157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:09.163671 2157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:09.168907 2163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:09.168912 2162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:09.168912 2165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:09.169189 2157 server_base.cc:1034] running on GCE node
I20260619 08:24:09.169399 2157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:09.169669 2157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:09.170842 2157 hybrid_clock.cc:648] HybridClock initialized: now 1781857449170816 us; error 40 us; skew 500 ppm
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857449, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:24:09.173414 2157 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:24:09.173904 2157 webserver.cc:460] Webserver started at http://127.25.127.131:34025/ using document root <none> and password file <none>
I20260619 08:24:09.174146 2157 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:09.174216 2157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:09.174371 2157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:09.175510 2157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "406ada65f1544ecdace5781f6e86f397"
format_stamp: "Formatted at 2026-06-19 08:24:09 on dist-test-slave-5rhg"
server_key: "8c822c915bf801250ae1938350b81526"
server_key_iv: "6ca193894b50526ce0bf9b682c591a8e"
server_key_version: "encryptionkey@0"
I20260619 08:24:09.175977 2157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "406ada65f1544ecdace5781f6e86f397"
format_stamp: "Formatted at 2026-06-19 08:24:09 on dist-test-slave-5rhg"
server_key: "8c822c915bf801250ae1938350b81526"
server_key_iv: "6ca193894b50526ce0bf9b682c591a8e"
server_key_version: "encryptionkey@0"
I20260619 08:24:09.177412 2157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:24:09.178154 2172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:09.178347 2157 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:24:09.178442 2157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "406ada65f1544ecdace5781f6e86f397"
format_stamp: "Formatted at 2026-06-19 08:24:09 on dist-test-slave-5rhg"
server_key: "8c822c915bf801250ae1938350b81526"
server_key_iv: "6ca193894b50526ce0bf9b682c591a8e"
server_key_version: "encryptionkey@0"
I20260619 08:24:09.178534 2157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:09.186509 2157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:09.188549 2157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:09.188701 2157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:09.188967 2157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:24:09.189285 2157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:24:09.189318 2157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:09.189342 2157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:24:09.189386 2157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:09.195439 2157 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:34701
I20260619 08:24:09.195483 2285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:34701 every 8 connection(s)
I20260619 08:24:09.195825 2157 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:24:09.201905 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 2157
I20260619 08:24:09.201996 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857449, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:24:09.202214 26110 external_mini_cluster.cc:1428] Setting key a6a806bb71d22b0f20cbb9a97a923f0c
I20260619 08:24:09.205663 1885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:09.196664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:58735 (local address 127.25.127.190:36493)
0619 08:24:09.196722 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:09.196725 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:09.196982 (+ 257us) server_negotiation.cc:407] Connection header received
0619 08:24:09.197554 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:09.197555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:09.197567 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:09.197628 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:09.198589 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.199123 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.200091 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.200347 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.200408 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.202675 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:09.202676 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:09.202676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:09.202684 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:09.204228 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:09.204666 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:09.204668 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:09.204668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:09.204733 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:09.204993 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:09.204994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:09.204994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:09.205117 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:09.205194 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:09.205487 (+ 293us) server_negotiation.cc:299] Negotiation successful
0619 08:24:09.205568 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:24:09.205871 2288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:09.196658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.131:58735)
0619 08:24:09.196837 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:09.196877 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:09.197435 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:09.197693 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:09.197697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:09.198290 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:09.198496 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.198501 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.199212 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.199213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.200005 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.200006 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.200424 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.200425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.200534 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.201027 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:09.201035 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:09.202584 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:09.204323 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:09.204331 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:09.204333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:09.204574 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:09.204800 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:09.204801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:09.204801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:09.204924 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:09.205185 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:09.205192 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:09.205341 (+ 149us) client_negotiation.cc:769] Sending connection context
0619 08:24:09.205453 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 08:24:09.205625 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260619 08:24:09.206317 2286 heartbeater.cc:347] Connected to a master server at 127.25.127.190:36493
I20260619 08:24:09.206424 2286 heartbeater.cc:464] Registering TS with master...
I20260619 08:24:09.206609 2286 heartbeater.cc:511] Master 127.25.127.190:36493 requested a full tablet report, sending...
I20260619 08:24:09.207041 1826 ts_manager.cc:194] Registered new tserver with Master: 406ada65f1544ecdace5781f6e86f397 (127.25.127.131:34701)
I20260619 08:24:09.207429 1826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:58735
I20260619 08:24:09.214052 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.129.keytab.
Entry for principal oryx/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.129.keytab.
WARNING: no policy specified for oryx/127.25.127.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.127.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.130.keytab.
Entry for principal oryx/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.130.keytab.
WARNING: no policy specified for oryx/127.25.127.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.131.keytab.
Entry for principal oryx/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.131.keytab.
I20260619 08:24:09.284878 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 1889
I20260619 08:24:09.289904 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 2023
I20260619 08:24:09.294499 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 2157
I20260619 08:24:09.299142 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 1794
I20260619 08:24:09.304697 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:36493
--webserver_interface=127.25.127.190
--webserver_port=34451
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:39963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:36493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:24:09.388375 2295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:09.388598 2295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:09.388623 2295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:09.389834 2295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:24:09.389878 2295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:09.389891 2295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:09.389904 2295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:24:09.389916 2295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:24:09.391485 2295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:39963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:36493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:36493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=34451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:09.391757 2295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:09.391997 2295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:09.397010 2303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:09.397010 2301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:09.397058 2300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:09.397266 2295 server_base.cc:1034] running on GCE node
I20260619 08:24:09.397514 2295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:09.397763 2295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:09.398907 2295 hybrid_clock.cc:648] HybridClock initialized: now 1781857449398892 us; error 40 us; skew 500 ppm
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857449, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:24:09.401607 2295 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:24:09.402081 2295 webserver.cc:460] Webserver started at http://127.25.127.190:34451/ using document root <none> and password file <none>
I20260619 08:24:09.402316 2295 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:09.402401 2295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:09.403738 2295 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:09.404476 2310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:09.404647 2295 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:09.404757 2295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "401091590ca84968a229cad6397e0f43"
format_stamp: "Formatted at 2026-06-19 08:24:08 on dist-test-slave-5rhg"
server_key: "dfe8cf7aa5a09647eb761cd9f7595a5f"
server_key_iv: "dacaee908914bc2463c4b3d0314a7b90"
server_key_version: "encryptionkey@0"
I20260619 08:24:09.405130 2295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:09.416219 2295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:09.418210 2295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:09.418386 2295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:09.422511 2295 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:36493
I20260619 08:24:09.422555 2362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:36493 every 8 connection(s)
I20260619 08:24:09.422971 2295 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:24:09.424503 2363 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:24:09.425173 2363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43: Bootstrap starting.
I20260619 08:24:09.429533 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 2295
I20260619 08:24:09.429842 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:46825
--local_ip_for_outbound_sockets=127.25.127.129
--tserver_master_addrs=127.25.127.190:36493
--webserver_port=37857
--webserver_interface=127.25.127.129
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:39963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.127.129
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.129.keytab with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:24:09.434713 2363 log.cc:824] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43: Log is configured to *not* fsync() on all Append() calls
I20260619 08:24:09.437270 2363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:24:09.437644 2363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43: Bootstrap complete.
I20260619 08:24:09.441740 2363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "401091590ca84968a229cad6397e0f43" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36493 } }
I20260619 08:24:09.442072 2363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 401091590ca84968a229cad6397e0f43, State: Initialized, Role: FOLLOWER
I20260619 08:24:09.442233 2363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "401091590ca84968a229cad6397e0f43" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36493 } }
I20260619 08:24:09.442302 2363 raft_consensus.cc:388] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:24:09.442340 2363 raft_consensus.cc:482] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:24:09.442379 2363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:24:09.443789 2363 raft_consensus.cc:504] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "401091590ca84968a229cad6397e0f43" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36493 } }
I20260619 08:24:09.443912 2363 leader_election.cc:302] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 401091590ca84968a229cad6397e0f43; no voters:
I20260619 08:24:09.444103 2363 leader_election.cc:288] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 08:24:09.444268 2367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 08:24:09.444424 2363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:24:09.444489 2367 raft_consensus.cc:686] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [term 2 LEADER]: Becoming Leader. State: Replica: 401091590ca84968a229cad6397e0f43, State: Running, Role: LEADER
I20260619 08:24:09.444672 2367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "401091590ca84968a229cad6397e0f43" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36493 } }
I20260619 08:24:09.445099 2369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 401091590ca84968a229cad6397e0f43. Latest consensus state: current_term: 2 leader_uuid: "401091590ca84968a229cad6397e0f43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "401091590ca84968a229cad6397e0f43" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36493 } } }
I20260619 08:24:09.445189 2369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [sys.catalog]: This master's current role is: LEADER
I20260619 08:24:09.445546 2368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "401091590ca84968a229cad6397e0f43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "401091590ca84968a229cad6397e0f43" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36493 } } }
I20260619 08:24:09.445619 2368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43 [sys.catalog]: This master's current role is: LEADER
I20260619 08:24:09.445910 2376 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:24:09.446585 2376 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:24:09.446753 2376 catalog_manager.cc:1259] Loaded cluster ID: 88fefa8b62074107bc06bc25d6763d74
I20260619 08:24:09.446825 2376 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:24:09.448038 2376 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:24:09.448299 2376 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 401091590ca84968a229cad6397e0f43: Loaded TSK: 0
I20260619 08:24:09.448598 2376 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 08:24:09.519035 2365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:09.519243 2365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:09.519266 2365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:09.520509 2365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:09.520558 2365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:09.520586 2365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:24:09.522074 2365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:39963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.129.keytab
--principal=oryx/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:46825
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=37857
--tserver_master_addrs=127.25.127.190:36493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:09.522328 2365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:09.522569 2365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:09.527508 2390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:09.527580 2389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:09.527606 2392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:09.528024 2365 server_base.cc:1034] running on GCE node
I20260619 08:24:09.528242 2365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:09.528496 2365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:09.529698 2365 hybrid_clock.cc:648] HybridClock initialized: now 1781857449529684 us; error 32 us; skew 500 ppm
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857449, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:24:09.532459 2365 init.cc:377] Logged in from keytab as oryx/127.25.127.129@KRBTEST.COM (short username oryx)
I20260619 08:24:09.532976 2365 webserver.cc:460] Webserver started at http://127.25.127.129:37857/ using document root <none> and password file <none>
I20260619 08:24:09.533231 2365 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:09.533326 2365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:09.534754 2365 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:24:09.535356 2399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:09.535502 2365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:09.535590 2365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "2908637f042d401f9ca9d6907379439b"
format_stamp: "Formatted at 2026-06-19 08:24:08 on dist-test-slave-5rhg"
server_key: "9284b2a414ae2e6236611ddc6a483a66"
server_key_iv: "0d7d68d84caae8e58d5a4eddc79ce23c"
server_key_version: "encryptionkey@0"
I20260619 08:24:09.535960 2365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:09.557523 2365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:09.559710 2365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:09.559891 2365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:09.560180 2365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:24:09.560572 2365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:24:09.560631 2365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:09.560678 2365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:24:09.560703 2365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:09.568260 2365 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:46825
I20260619 08:24:09.568341 2512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:46825 every 8 connection(s)
I20260619 08:24:09.568707 2365 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:24:09.576012 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 2365
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:09.576327 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:34175
--local_ip_for_outbound_sockets=127.25.127.130
--tserver_master_addrs=127.25.127.190:36493
--webserver_port=46103
--webserver_interface=127.25.127.130
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:39963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.127.130
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.130.keytab with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:09.577692 2515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:09.569489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:40851)
0619 08:24:09.569668 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:09.569713 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:09.570195 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:09.570724 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:09.570728 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:09.571342 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:09.571565 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.571571 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.573173 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.573174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.573986 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.573989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.574588 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.574589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.574691 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.575188 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:09.575196 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:09.577475 (+ 2279us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
W20260619 08:24:09.577739 2516 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:09.569555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:40851 (local address 127.25.127.190:36493)
0619 08:24:09.569757 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:09.569767 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:09.569804 (+ 37us) server_negotiation.cc:407] Connection header received
0619 08:24:09.570407 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:09.570415 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:09.570475 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:09.570653 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:09.571700 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.573071 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.574070 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.574504 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.574588 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.577593 (+ 3005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.129:40851: BlockingRecv error: recv got EOF from 127.25.127.129:40851 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":87,"threads_started":1}
W20260619 08:24:09.578105 2513 heartbeater.cc:644] Failed to heartbeat to 127.25.127.190:36493 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.127.190:36493: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:09.582924 2515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:09.578404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:57439)
0619 08:24:09.578450 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:09.578464 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:09.578520 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:09.578645 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:09.578645 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:09.578782 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:09.578823 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.578824 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.579464 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.579465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.579967 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.579968 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.580438 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.580439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.580512 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.580974 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:09.580979 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:09.582819 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 08:24:09.582993 2516 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:09.578468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:57439 (local address 127.25.127.190:36493)
0619 08:24:09.578527 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:09.578529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:09.578554 (+ 25us) server_negotiation.cc:407] Connection header received
0619 08:24:09.578575 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:09.578576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:09.578587 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:09.578640 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:09.578905 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.579380 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.580066 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.580354 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.580438 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.582947 (+ 2509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.129:57439: BlockingRecv error: recv got EOF from 127.25.127.129:57439 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:09.587817 2515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:09.583281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:54029)
0619 08:24:09.583357 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:09.583365 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:09.583429 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:09.583577 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:09.583578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:09.583710 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:09.583738 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.583739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.584435 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.584436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.584933 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.584935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.585283 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.585284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.585337 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.585846 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:09.585850 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:09.587737 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260619 08:24:09.587944 2516 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:09.583371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:54029 (local address 127.25.127.190:36493)
0619 08:24:09.583434 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:09.583437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:09.583445 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:24:09.583489 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:09.583490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:09.583503 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:09.583574 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:09.583819 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.584353 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.585029 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.585211 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.585286 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.587843 (+ 2557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.129:54029: BlockingRecv error: recv got EOF from 127.25.127.129:54029 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260619 08:24:09.587981 2513 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 08:24:09.662026 2517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:09.662250 2517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:09.662278 2517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:09.663532 2517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:09.663582 2517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:09.663610 2517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:24:09.665149 2517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:39963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.130.keytab
--principal=oryx/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:34175
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=46103
--tserver_master_addrs=127.25.127.190:36493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:09.665426 2517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:09.665674 2517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:09.670976 2523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:09.670959 2522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:09.671312 2517 server_base.cc:1034] running on GCE node
W20260619 08:24:09.670959 2525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:09.671657 2517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:09.671953 2517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:09.673111 2517 hybrid_clock.cc:648] HybridClock initialized: now 1781857449673101 us; error 27 us; skew 500 ppm
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857449, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:24:09.675915 2517 init.cc:377] Logged in from keytab as oryx/127.25.127.130@KRBTEST.COM (short username oryx)
I20260619 08:24:09.676417 2517 webserver.cc:460] Webserver started at http://127.25.127.130:46103/ using document root <none> and password file <none>
I20260619 08:24:09.676671 2517 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:09.676754 2517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:09.678175 2517 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:09.678905 2532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:09.679138 2517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:09.679252 2517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "656973288a8b4a06a92a9a77def54218"
format_stamp: "Formatted at 2026-06-19 08:24:09 on dist-test-slave-5rhg"
server_key: "8718b8661a0a56e4be1d6548466ff741"
server_key_iv: "b6c6a09b7d943ebec2ad1adcc8a2c766"
server_key_version: "encryptionkey@0"
I20260619 08:24:09.679637 2517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:09.704311 2517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:09.706544 2517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:09.706719 2517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:09.706979 2517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:24:09.707373 2517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:24:09.707415 2517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:09.707455 2517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:24:09.707502 2517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:09.713984 2517 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:34175
I20260619 08:24:09.714056 2645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:34175 every 8 connection(s)
I20260619 08:24:09.714411 2517 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:09.721868 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 2517
I20260619 08:24:09.722193 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:34701
--local_ip_for_outbound_sockets=127.25.127.131
--tserver_master_addrs=127.25.127.190:36493
--webserver_port=34025
--webserver_interface=127.25.127.131
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:39963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.127.131
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.131.keytab with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:24:09.722220 2648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:09.715129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:43127)
0619 08:24:09.715342 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:09.715395 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:09.715893 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:09.716293 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:09.716298 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:09.716940 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:09.717157 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.717161 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.717860 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.717861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.718692 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.718694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.719096 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.719097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.719207 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.719731 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:09.719740 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:09.722057 (+ 2317us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":90,"threads_started":1}
W20260619 08:24:09.722257 2516 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:09.715210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:43127 (local address 127.25.127.190:36493)
0619 08:24:09.715283 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:09.715285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:09.715531 (+ 246us) server_negotiation.cc:407] Connection header received
0619 08:24:09.716050 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:09.716052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:09.716066 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:09.716134 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:09.717268 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.717772 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.718795 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.719010 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.719077 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.722187 (+ 3110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.130:43127: BlockingRecv error: recv got EOF from 127.25.127.130:43127 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260619 08:24:09.722771 2646 heartbeater.cc:644] Failed to heartbeat to 127.25.127.190:36493 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.127.190:36493: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:09.727706 2648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:09.723045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:40583)
0619 08:24:09.723096 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:09.723104 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:09.723157 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:09.723314 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:09.723315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:09.723449 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:09.723496 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.723498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.724146 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.724167 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.724676 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.724678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.725178 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.725179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.725245 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.725732 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:09.725736 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:09.727601 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 08:24:09.727741 2516 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:09.723103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:40583 (local address 127.25.127.190:36493)
0619 08:24:09.723168 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:09.723170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:09.723179 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:24:09.723218 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:09.723219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:09.723228 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:09.723297 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:09.723583 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.724063 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.724763 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.725070 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.725141 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.727694 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.130:40583: BlockingRecv error: recv got EOF from 127.25.127.130:40583 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:09.733351 2648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:09.727994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:58329)
0619 08:24:09.728139 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:09.728172 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:09.728216 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:09.728484 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:09.728485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:09.728643 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:09.728686 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.728688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.729349 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.729350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.730250 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.730253 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.730632 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.730633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.730698 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.731172 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:09.731177 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:09.733257 (+ 2080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
W20260619 08:24:09.733407 2516 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:09.728299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:58329 (local address 127.25.127.190:36493)
0619 08:24:09.728357 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:09.728361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:09.728373 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:24:09.728393 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:09.728394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:09.728401 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:09.728450 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:09.728768 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.729267 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.730351 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.730536 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.730621 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.733347 (+ 2726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.130:58329: BlockingRecv error: recv got EOF from 127.25.127.130:58329 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260619 08:24:09.733551 2646 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 08:24:09.807440 2649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:09.807713 2649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:09.807754 2649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:09.809113 2649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:09.809180 2649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:09.809226 2649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:24:09.810721 2649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:39963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/krb5kdc/oryx_127.25.127.131.keytab
--principal=oryx/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:34701
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=34025
--tserver_master_addrs=127.25.127.190:36493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:09.811009 2649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:09.811256 2649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:09.816449 2656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:09.816540 2658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:09.816468 2655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:09.816581 2649 server_base.cc:1034] running on GCE node
I20260619 08:24:09.816872 2649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:09.817127 2649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:09.818284 2649 hybrid_clock.cc:648] HybridClock initialized: now 1781857449818256 us; error 41 us; skew 500 ppm
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857449, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:24:09.820991 2649 init.cc:377] Logged in from keytab as oryx/127.25.127.131@KRBTEST.COM (short username oryx)
I20260619 08:24:09.821437 2649 webserver.cc:460] Webserver started at http://127.25.127.131:34025/ using document root <none> and password file <none>
I20260619 08:24:09.821667 2649 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:09.821748 2649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:09.823118 2649 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:24:09.823796 2665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:09.824106 2649 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:24:09.824245 2649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "406ada65f1544ecdace5781f6e86f397"
format_stamp: "Formatted at 2026-06-19 08:24:09 on dist-test-slave-5rhg"
server_key: "8c822c915bf801250ae1938350b81526"
server_key_iv: "6ca193894b50526ce0bf9b682c591a8e"
server_key_version: "encryptionkey@0"
I20260619 08:24:09.824635 2649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:09.835810 2649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:09.838142 2649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:09.838307 2649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:09.838558 2649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:24:09.838900 2649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:24:09.838964 2649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:09.838999 2649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:24:09.839020 2649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:09.845042 2649 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:34701
I20260619 08:24:09.845095 2778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:34701 every 8 connection(s)
I20260619 08:24:09.845453 2649 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:24:09.847916 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 2649
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 08:24:09.853814 2516 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:09.846186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:36415 (local address 127.25.127.190:36493)
0619 08:24:09.846264 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:09.846266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:09.846508 (+ 242us) server_negotiation.cc:407] Connection header received
0619 08:24:09.847059 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:09.847060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:09.847073 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:09.847133 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:09.848167 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.848852 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.850208 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.850503 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.850593 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.853728 (+ 3135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.131:36415: BlockingRecv error: recv got EOF from 127.25.127.131:36415 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 08:24:09.853919 2781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:09.846185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.131:36415)
0619 08:24:09.846372 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:09.846409 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:09.846942 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:09.847218 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:09.847222 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:09.847803 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:09.848009 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.848014 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.848942 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.848943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.850092 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.850094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.850604 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.850605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.850713 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.851285 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:09.851295 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:09.853649 (+ 2354us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":95,"threads_started":1}
W20260619 08:24:09.854341 2779 heartbeater.cc:644] Failed to heartbeat to 127.25.127.190:36493 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.127.190:36493: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
I20260619 08:24:09.856230 2783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:09.848509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41166 (local address 127.25.127.190:36493)
0619 08:24:09.848621 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:09.848623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:09.848631 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:24:09.848685 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:09.848686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:09.848695 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:09.848748 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:09.849398 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.850123 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.851123 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.851427 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.851505 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.852772 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:09.852777 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:09.852777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:09.852790 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:09.854780 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:09.855146 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:09.855150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:09.855151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:09.855256 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:09.855465 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:09.855466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:09.855466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:09.855704 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:09.855858 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:09.855974 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 08:24:09.856093 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":73,"threads_started":1}
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:09.858994 2781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:09.854605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.131:44721)
0619 08:24:09.854657 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:09.854668 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:09.854711 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:09.854824 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:09.854824 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:09.854959 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:09.855035 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.855037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.855681 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.855682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.856295 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.856299 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.856711 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.856712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.856767 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.857237 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:09.857243 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:09.858936 (+ 1693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 08:24:09.859099 2516 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:09.854668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:44721 (local address 127.25.127.190:36493)
0619 08:24:09.854730 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:09.854732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:09.854740 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:24:09.854761 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:09.854763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:09.854772 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:09.854822 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:09.855122 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.855593 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.856380 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.856640 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.856697 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.859029 (+ 2332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.131:44721: BlockingRecv error: recv got EOF from 127.25.127.131:44721 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:09 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:09.863266 2781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:09.859287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.131:56975)
0619 08:24:09.859373 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:09.859383 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:09.859413 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:09.859519 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:09.859520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:09.859624 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:09.859652 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.859653 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.860299 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.860300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.860777 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:09.860778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.861098 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.861098 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:09.861136 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.861483 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:09.861487 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:09.863180 (+ 1693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260619 08:24:09.863313 2516 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:09.859334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:56975 (local address 127.25.127.190:36493)
0619 08:24:09.859394 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:09.859396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:09.859415 (+ 19us) server_negotiation.cc:407] Connection header received
0619 08:24:09.859458 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:09.859459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:09.859467 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:09.859504 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:09.859708 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.860233 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.860844 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:09.861036 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:09.861085 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:09.863249 (+ 2164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.131:56975: BlockingRecv error: recv got EOF from 127.25.127.131:56975 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260619 08:24:09.863487 2779 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 08:24:10 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:10 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:10.593202 2784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:10.588453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:33351)
0619 08:24:10.588615 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:10.588626 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:10.588679 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:10.588807 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:10.588808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:10.588996 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:10.589047 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:10.589049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:10.589728 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:10.589729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:10.590321 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:10.590322 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:10.590715 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:10.590716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:10.590773 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:10.591251 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:10.591256 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:10.593135 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
W20260619 08:24:10.593369 2785 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:10.588557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:33351 (local address 127.25.127.190:36493)
0619 08:24:10.588682 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:10.588684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:10.588695 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:24:10.588720 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:10.588721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:10.588730 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:10.588789 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:10.589118 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:10.589631 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:10.590416 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:10.590622 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:10.590679 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:10.593265 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.129:33351: BlockingRecv error: recv got EOF from 127.25.127.129:33351 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":53,"threads_started":1}
Jun 19 08:24:10 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:10 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:10.738869 2786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:10.733939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:49143)
0619 08:24:10.734110 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:10.734121 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:10.734190 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:10.734320 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:10.734321 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:10.734488 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:10.734544 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:10.734546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:10.735194 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:10.735195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:10.735759 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:10.735760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:10.736106 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:10.736107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:10.736172 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:10.736652 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:10.736657 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:10.738743 (+ 2086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":85,"threads_started":1}
W20260619 08:24:10.738909 2785 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:10.734052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:49143 (local address 127.25.127.190:36493)
0619 08:24:10.734105 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:10.734107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:10.734187 (+ 80us) server_negotiation.cc:407] Connection header received
0619 08:24:10.734235 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:10.734236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:10.734246 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:10.734296 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:10.734595 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:10.735114 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:10.735838 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:10.736037 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:10.736089 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:10.738841 (+ 2752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.130:49143: BlockingRecv error: recv got EOF from 127.25.127.130:49143 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Jun 19 08:24:10 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:10 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:10.868866 2787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:10.864007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.131:44143)
0619 08:24:10.864172 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:10.864183 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:10.864233 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:10.864398 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:10.864398 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:10.864561 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:10.864612 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:10.864614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:10.865283 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:10.865285 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:10.865850 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:10.865851 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:10.866242 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:10.866243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:10.866284 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:10.866732 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:10.866736 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:10.868756 (+ 2020us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
W20260619 08:24:10.868953 2785 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:10.864098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:44143 (local address 127.25.127.190:36493)
0619 08:24:10.864201 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:10.864203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:10.864218 (+ 15us) server_negotiation.cc:407] Connection header received
0619 08:24:10.864304 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:10.864305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:10.864317 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:10.864378 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:10.864680 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:10.865196 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:10.865972 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:10.866169 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:10.866233 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:10.868873 (+ 2640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.131:44143: BlockingRecv error: recv got EOF from 127.25.127.131:44143 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Jun 19 08:24:11 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:11 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:11.598536 2788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:11.593710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:53729)
0619 08:24:11.593853 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:11.593865 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:11.593928 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:11.594085 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:11.594086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:11.594318 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:11.594369 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:11.594371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:11.595099 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:11.595100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:11.595645 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:11.595647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:11.596048 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:11.596049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:11.596118 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:11.596593 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:11.596597 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:11.598474 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":124,"threads_started":1}
W20260619 08:24:11.598675 2789 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:11.593814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:53729 (local address 127.25.127.190:36493)
0619 08:24:11.593946 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:11.593948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:11.593957 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:24:11.593981 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:11.593982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:11.593993 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:11.594059 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:11.594461 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:11.595004 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:11.595729 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:11.595958 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:11.596048 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:11.598564 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.129:53729: BlockingRecv error: recv got EOF from 127.25.127.129:53729 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":82,"threads_started":1}
Jun 19 08:24:11 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:11 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:11.744102 2790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:11.739313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:46385)
0619 08:24:11.739484 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:11.739495 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:11.739562 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:11.739717 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:11.739718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:11.739981 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:11.740029 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:11.740031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:11.740833 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:11.740834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:11.741379 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:11.741381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:11.741767 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:11.741768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:11.741826 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:11.742250 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:11.742255 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:11.744041 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
W20260619 08:24:11.744280 2789 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:11.739379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:46385 (local address 127.25.127.190:36493)
0619 08:24:11.739469 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:11.739472 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:11.739541 (+ 69us) server_negotiation.cc:407] Connection header received
0619 08:24:11.739616 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:11.739617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:11.739630 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:11.739698 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:11.740109 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:11.740749 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:11.741478 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:11.741685 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:11.741756 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:11.744212 (+ 2456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.130:46385: BlockingRecv error: recv got EOF from 127.25.127.130:46385 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Jun 19 08:24:11 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:11 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:11.874116 2791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:11.869347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.131:35103)
0619 08:24:11.869538 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:11.869551 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:11.869632 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:11.869767 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:11.869768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:11.869926 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:11.869978 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:11.869980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:11.870641 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:11.870642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:11.871384 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:11.871387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:11.871771 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:11.871772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:11.871824 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:11.872351 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:11.872357 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:11.874052 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":101,"threads_started":1}
W20260619 08:24:11.874245 2789 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:11.869524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:35103 (local address 127.25.127.190:36493)
0619 08:24:11.869577 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:11.869580 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:11.869616 (+ 36us) server_negotiation.cc:407] Connection header received
0619 08:24:11.869678 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:11.869678 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:11.869691 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:11.869746 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:11.870053 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:11.870565 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:11.871480 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:11.871680 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:11.871742 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:11.874142 (+ 2400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.131:35103: BlockingRecv error: recv got EOF from 127.25.127.131:35103 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Jun 19 08:24:12 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:12 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:12.603634 2792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:12.599042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:55061)
0619 08:24:12.599154 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:12.599172 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:12.599220 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:12.599347 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:12.599348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:12.599514 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:12.599566 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:12.599567 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:12.600268 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:12.600268 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:12.600788 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:12.600789 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:12.601183 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:12.601184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:12.601224 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:12.601653 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:12.601657 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:12.603521 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":65,"threads_started":1}
W20260619 08:24:12.603684 2793 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:12.599073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:55061 (local address 127.25.127.190:36493)
0619 08:24:12.599204 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:12.599206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:12.599215 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:24:12.599264 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:12.599264 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:12.599276 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:12.599336 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:12.599628 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:12.600197 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:12.600885 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:12.601098 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:12.601168 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:12.603610 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.129:55061: BlockingRecv error: recv got EOF from 127.25.127.129:55061 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
Jun 19 08:24:12 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:12 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:12.749390 2794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:12.744645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:42891)
0619 08:24:12.744821 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:12.744832 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:12.744929 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:12.745121 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:12.745123 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:12.745301 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:12.745353 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:12.745355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:12.746086 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:12.746086 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:12.746692 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:12.746694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:12.747047 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:12.747048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:12.747087 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:12.747558 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:12.747563 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:12.749316 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":101,"threads_started":1}
W20260619 08:24:12.749567 2793 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:12.744692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:42891 (local address 127.25.127.190:36493)
0619 08:24:12.744752 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:12.744754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:12.744907 (+ 153us) server_negotiation.cc:407] Connection header received
0619 08:24:12.744993 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:12.744994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:12.745012 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:12.745067 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:12.745427 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:12.746022 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:12.746765 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:12.746981 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:12.747037 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:12.749466 (+ 2429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.130:42891: BlockingRecv error: recv got EOF from 127.25.127.130:42891 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Jun 19 08:24:12 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:12 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:12.879056 2795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:12.874584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.131:55619)
0619 08:24:12.874735 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:12.874748 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:12.874808 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:12.874941 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:12.874942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:12.875107 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:12.875160 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:12.875162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:12.875802 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:12.875803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:12.876374 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:12.876375 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:12.876703 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:12.876703 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:12.876750 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:12.877211 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:12.877215 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:12.878984 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":100,"threads_started":1}
W20260619 08:24:12.879192 2793 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:12.874641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:55619 (local address 127.25.127.190:36493)
0619 08:24:12.874707 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:12.874709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:12.874807 (+ 98us) server_negotiation.cc:407] Connection header received
0619 08:24:12.874851 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:12.874851 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:12.874864 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:12.874917 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:12.875227 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:12.875726 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:12.876442 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:12.876630 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:12.876688 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:12.879106 (+ 2418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.131:55619: BlockingRecv error: recv got EOF from 127.25.127.131:55619 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 08:24:13 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:13 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:13.609084 2796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:13.604196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:50285)
0619 08:24:13.604331 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:13.604349 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:13.604399 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:13.604548 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:13.604549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:13.604715 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:13.604768 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:13.604769 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:13.605416 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:13.605417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:13.605939 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:13.605941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:13.606386 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:13.606387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:13.606428 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:13.606942 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:13.606947 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:13.608960 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
W20260619 08:24:13.609146 2797 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:13.604288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:50285 (local address 127.25.127.190:36493)
0619 08:24:13.604416 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:13.604418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:13.604428 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:24:13.604451 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:13.604451 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:13.604461 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:13.604534 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:13.604831 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:13.605347 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:13.606099 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:13.606312 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:13.606376 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:13.609065 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.129:50285: BlockingRecv error: recv got EOF from 127.25.127.129:50285 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":73,"threads_started":1}
Jun 19 08:24:13 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:13 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:13.755076 2798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:13.749975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:38359)
0619 08:24:13.750177 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:13.750189 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:13.750292 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:13.750460 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:13.750461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:13.750655 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:13.750705 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:13.750707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:13.751526 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:13.751527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:13.752105 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:13.752108 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:13.752556 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:13.752557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:13.752617 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:13.753096 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:13.753100 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:13.754972 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":109,"threads_started":1}
W20260619 08:24:13.755125 2797 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:13.750074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:38359 (local address 127.25.127.190:36493)
0619 08:24:13.750167 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:13.750170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:13.750271 (+ 101us) server_negotiation.cc:407] Connection header received
0619 08:24:13.750356 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:13.750357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:13.750369 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:13.750440 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:13.750789 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:13.751436 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:13.752247 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:13.752466 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:13.752555 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:13.755061 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.130:38359: BlockingRecv error: recv got EOF from 127.25.127.130:38359 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 19 08:24:13 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:13 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:13.884536 2799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:13.879566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.131:37533)
0619 08:24:13.879723 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:13.879742 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:13.879800 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:13.879938 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:13.879939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:13.880104 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:13.880175 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:13.880177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:13.880898 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:13.880899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:13.881437 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:13.881438 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:13.881777 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:13.881778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:13.881825 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:13.882361 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:13.882366 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:13.884469 (+ 2103us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":67,"threads_started":1}
W20260619 08:24:13.884682 2797 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:13.879680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:37533 (local address 127.25.127.190:36493)
0619 08:24:13.879776 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:13.879778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:13.879787 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:24:13.879857 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:13.879858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:13.879870 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:13.879921 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:13.880271 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:13.880807 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:13.881515 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:13.881699 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:13.881756 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:13.884585 (+ 2829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.131:37533: BlockingRecv error: recv got EOF from 127.25.127.131:37533 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Jun 19 08:24:14 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:14 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:14.614943 2800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:14.609645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:50267)
0619 08:24:14.609843 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:14.609855 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:14.609929 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:14.610160 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:14.610162 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:14.610329 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:14.610405 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:14.610406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:14.611131 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:14.611132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:14.611676 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:14.611677 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:14.612070 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:14.612071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:14.612125 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:14.612953 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:14.612959 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:14.614878 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":122,"threads_started":1}
W20260619 08:24:14.615077 2801 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:14.609711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:50267 (local address 127.25.127.190:36493)
0619 08:24:14.609860 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:14.609862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:14.609914 (+ 52us) server_negotiation.cc:407] Connection header received
0619 08:24:14.609992 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:14.609992 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:14.610003 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:14.610066 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:14.610487 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:14.611035 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:14.611763 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:14.611966 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:14.612025 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:14.614988 (+ 2963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.129:50267: BlockingRecv error: recv got EOF from 127.25.127.129:50267 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
Jun 19 08:24:14 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:14 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:14.764676 2802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:14.755722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:52989)
0619 08:24:14.755982 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:14.755993 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:14.756047 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:14.756248 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:14.756249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:14.756421 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:14.756477 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:14.756479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:14.757208 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:14.757210 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:14.757805 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:14.757807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:14.758174 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:14.758175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:14.758222 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:14.762688 (+ 4466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:14.762693 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:14.764613 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":65,"threads_started":1}
W20260619 08:24:14.764827 2801 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:14.755999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:52989 (local address 127.25.127.190:36493)
0619 08:24:14.756075 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:14.756077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:14.756087 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:24:14.756115 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:14.756116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:14.756126 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:14.756233 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:14.756552 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:14.757103 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:14.757904 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:14.758101 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:14.758160 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:14.764738 (+ 6578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.130:52989: BlockingRecv error: recv got EOF from 127.25.127.130:52989 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 19 08:24:14 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:14 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:14.889763 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:14.885129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.131:60665)
0619 08:24:14.885312 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:14.885323 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:14.885390 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:14.885519 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:14.885520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:14.885688 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:14.885741 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:14.885743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:14.886416 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:14.886417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:14.886958 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:14.886960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:14.887281 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:14.887281 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:14.887322 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:14.887867 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:14.887872 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:14.889701 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":97,"threads_started":1}
W20260619 08:24:14.889951 2801 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:14.885183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:60665 (local address 127.25.127.190:36493)
0619 08:24:14.885248 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:14.885250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:14.885387 (+ 137us) server_negotiation.cc:407] Connection header received
0619 08:24:14.885433 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:14.885433 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:14.885446 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:14.885494 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:14.885828 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:14.886341 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:14.887031 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:14.887214 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:14.887272 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:14.889849 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.131:60665: BlockingRecv error: recv got EOF from 127.25.127.131:60665 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 08:24:15 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:15 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:15.620472 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:15.615566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:46173)
0619 08:24:15.615730 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:15.615742 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:15.615802 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:15.615947 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:15.615948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:15.616111 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:15.616185 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:15.616187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:15.616910 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:15.616911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:15.617471 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:15.617473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:15.617850 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:15.617851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:15.617895 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:15.618428 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:15.618433 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:15.620409 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
W20260619 08:24:15.620664 2805 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:15.615677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:46173 (local address 127.25.127.190:36493)
0619 08:24:15.615818 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:15.615820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:15.615829 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:24:15.615853 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:15.615853 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:15.615863 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:15.615923 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:15.616286 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:15.616815 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:15.617569 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:15.617773 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:15.617838 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:15.620576 (+ 2738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.129:46173: BlockingRecv error: recv got EOF from 127.25.127.129:46173 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
Jun 19 08:24:15 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:15 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:15.770238 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:15.765358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:45157)
0619 08:24:15.765511 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:15.765524 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:15.765623 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:15.765795 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:15.765796 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:15.766042 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:15.766101 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:15.766103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:15.766772 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:15.766773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:15.767330 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:15.767332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:15.767667 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:15.767667 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:15.767708 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:15.768208 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:15.768214 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:15.770176 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
W20260619 08:24:15.770483 2805 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:15.765443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:45157 (local address 127.25.127.190:36493)
0619 08:24:15.765506 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:15.765508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:15.765601 (+ 93us) server_negotiation.cc:407] Connection header received
0619 08:24:15.765696 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:15.765697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:15.765710 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:15.765778 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:15.766172 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:15.766675 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:15.767410 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:15.767601 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:15.767655 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:15.770308 (+ 2653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.130:45157: BlockingRecv error: recv got EOF from 127.25.127.130:45157 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 19 08:24:15 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:15 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:15.895088 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:15.890295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.131:37887)
0619 08:24:15.890471 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:15.890488 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:15.890541 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:15.890766 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:15.890768 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:15.890950 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:15.891002 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:15.891004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:15.891678 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:15.891679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:15.892259 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:15.892261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:15.892589 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:15.892590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:15.892633 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:15.893097 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:15.893102 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:15.894984 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":102,"threads_started":1}
W20260619 08:24:15.895128 2805 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:15.890422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:37887 (local address 127.25.127.190:36493)
0619 08:24:15.890513 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:15.890515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:15.890529 (+ 14us) server_negotiation.cc:407] Connection header received
0619 08:24:15.890638 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:15.890640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:15.890655 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:15.890716 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:15.891088 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:15.891599 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:15.892338 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:15.892522 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:15.892577 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:15.895067 (+ 2490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.131:37887: BlockingRecv error: recv got EOF from 127.25.127.131:37887 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 19 08:24:16 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:16 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:16.626040 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:16.620940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:49557)
0619 08:24:16.621085 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:16.621095 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:16.621154 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:16.621279 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:16.621280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:16.621443 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:16.621495 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:16.621497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:16.622212 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:16.622213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:16.622748 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:16.622749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:16.623275 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:16.623276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:16.623340 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:16.623994 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:16.623999 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:16.625966 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
W20260619 08:24:16.626232 2809 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:16.621021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:49557 (local address 127.25.127.190:36493)
0619 08:24:16.621139 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:16.621141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:16.621151 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:24:16.621203 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:16.621204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:16.621215 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:16.621275 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:16.621561 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:16.622119 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:16.622854 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:16.623178 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:16.623262 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:16.626114 (+ 2852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.129:49557: BlockingRecv error: recv got EOF from 127.25.127.129:49557 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":52,"threads_started":1}
Jun 19 08:24:16 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:16 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:16.776139 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:16.770880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:55401)
0619 08:24:16.771055 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:16.771066 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:16.771121 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:16.771340 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:16.771341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:16.771531 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:16.771589 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:16.771591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:16.772331 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:16.772332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:16.772900 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:16.772901 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:16.773239 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:16.773239 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:16.773281 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:16.773757 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:16.773761 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:16.776039 (+ 2278us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":107,"threads_started":1}
W20260619 08:24:16.776224 2809 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:16.770999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:55401 (local address 127.25.127.190:36493)
0619 08:24:16.771075 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:16.771077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:16.771110 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:24:16.771186 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:16.771187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:16.771203 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:16.771249 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:16.771693 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:16.772237 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:16.772981 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:16.773170 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:16.773224 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:16.776136 (+ 2912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.130:55401: BlockingRecv error: recv got EOF from 127.25.127.130:55401 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 19 08:24:16 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:16 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:16.900800 2811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:16.895680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.131:40755)
0619 08:24:16.895910 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:16.895923 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:16.896022 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:16.896262 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:16.896263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:16.896445 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:16.896494 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:16.896496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:16.897198 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:16.897199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:16.897754 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:16.897756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:16.898115 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:16.898116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:16.898182 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:16.898630 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:16.898634 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:16.900696 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":66,"threads_started":1}
W20260619 08:24:16.900856 2809 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:16.895915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:40755 (local address 127.25.127.190:36493)
0619 08:24:16.895985 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:16.895987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:16.896001 (+ 14us) server_negotiation.cc:407] Connection header received
0619 08:24:16.896105 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:16.896107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:16.896121 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:16.896249 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:16.896583 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:16.897113 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:16.897843 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:16.898039 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:16.898116 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:16.900785 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.131:40755: BlockingRecv error: recv got EOF from 127.25.127.131:40755 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 19 08:24:17 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:17 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:17.631664 2812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:17.626621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:52229)
0619 08:24:17.626813 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:17.626825 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:17.626891 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:17.627193 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:17.627194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:17.627461 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:17.627523 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:17.627526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:17.628270 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:17.628270 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:17.628879 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:17.628881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:17.629266 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:17.629267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:17.629307 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:17.629767 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:17.629771 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:17.631602 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":93,"threads_started":1}
W20260619 08:24:17.631842 2813 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:17.626901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:52229 (local address 127.25.127.190:36493)
0619 08:24:17.627031 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:17.627033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:17.627043 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:24:17.627068 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:17.627069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:17.627081 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:17.627168 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:17.627602 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:17.628201 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:17.628985 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:17.629185 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:17.629260 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:17.631733 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.129:52229: BlockingRecv error: recv got EOF from 127.25.127.129:52229 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
Jun 19 08:24:17 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:17 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:17.781863 2814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:17.776768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:34537)
0619 08:24:17.776973 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:17.776985 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:17.777055 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:17.777194 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:17.777195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:17.777358 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:17.777426 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:17.777427 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:17.778224 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:17.778225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:17.778768 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:17.778770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:17.779148 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:17.779149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:17.779201 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:17.779672 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:17.779676 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:17.781753 (+ 2077us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":129,"threads_started":1}
W20260619 08:24:17.781991 2813 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:17.776846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:34537 (local address 127.25.127.190:36493)
0619 08:24:17.776920 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:17.776922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:17.777056 (+ 134us) server_negotiation.cc:407] Connection header received
0619 08:24:17.777117 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:17.777118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:17.777129 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:17.777172 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:17.777493 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:17.778147 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:17.778864 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:17.779055 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:17.779108 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:17.781924 (+ 2816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.130:34537: BlockingRecv error: recv got EOF from 127.25.127.130:34537 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 19 08:24:17 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:17 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:17.906097 2815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:17.901403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.131:57815)
0619 08:24:17.901561 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:17.901575 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:17.901672 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:17.901858 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:17.901859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:17.902027 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:17.902086 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:17.902088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:17.902730 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:17.902731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:17.903281 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:17.903282 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:17.903619 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:17.903620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:17.903661 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:17.904123 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:17.904128 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:17.906018 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
W20260619 08:24:17.906335 2813 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:17.901463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:57815 (local address 127.25.127.190:36493)
0619 08:24:17.901522 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:17.901524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:17.901672 (+ 148us) server_negotiation.cc:407] Connection header received
0619 08:24:17.901732 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:17.901733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:17.901747 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:17.901807 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:17.902155 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:17.902652 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:17.903362 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:17.903550 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:17.903609 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:17.906259 (+ 2650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.131:57815: BlockingRecv error: recv got EOF from 127.25.127.131:57815 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Jun 19 08:24:18 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:18 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:18.637239 2816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:18.632234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:53547)
0619 08:24:18.632405 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:18.632417 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:18.632501 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:18.632634 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:18.632635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:18.632831 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:18.632884 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:18.632885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:18.633533 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:18.633534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:18.634110 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:18.634112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:18.634447 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:18.634448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:18.634494 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:18.634993 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:18.634997 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:18.637178 (+ 2181us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":62,"threads_started":1}
W20260619 08:24:18.637400 2817 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:18.632347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:53547 (local address 127.25.127.190:36493)
0619 08:24:18.632506 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:18.632508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:18.632519 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:24:18.632546 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:18.632547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:18.632559 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:18.632618 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:18.632956 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:18.633466 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:18.634187 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:18.634382 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:18.634432 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:18.637308 (+ 2876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.129:53547: BlockingRecv error: recv got EOF from 127.25.127.129:53547 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
Jun 19 08:24:18 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:18 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:18.787532 2818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:18.782524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:46703)
0619 08:24:18.782690 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:18.782701 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:18.782789 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:18.782935 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:18.782937 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:18.783184 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:18.783250 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:18.783252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:18.784045 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:18.784046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:18.784616 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:18.784617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:18.784998 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:18.784999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:18.785054 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:18.785525 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:18.785529 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:18.787459 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":77,"threads_started":1}
W20260619 08:24:18.787667 2817 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:18.782573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:46703 (local address 127.25.127.190:36493)
0619 08:24:18.782631 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:18.782633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:18.782797 (+ 164us) server_negotiation.cc:407] Connection header received
0619 08:24:18.782858 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:18.782859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:18.782872 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:18.782927 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:18.783342 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:18.783952 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:18.784713 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:18.784921 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:18.785001 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:18.787580 (+ 2579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.130:46703: BlockingRecv error: recv got EOF from 127.25.127.130:46703 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Jun 19 08:24:18 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:18 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:18.911911 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:18.906795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.131:58473)
0619 08:24:18.906946 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:18.906964 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:18.907021 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:18.907201 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:18.907202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:18.907455 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:18.907507 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:18.907510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:18.908279 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:18.908280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:18.908930 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:18.908933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:18.909419 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:18.909420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:18.909466 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:18.909933 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:18.909938 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:18.911846 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
W20260619 08:24:18.912052 2817 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:18.906878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:58473 (local address 127.25.127.190:36493)
0619 08:24:18.906942 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:18.906944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:18.907027 (+ 83us) server_negotiation.cc:407] Connection header received
0619 08:24:18.907067 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:18.907104 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:18.907121 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:18.907191 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:18.907573 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:18.908195 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:18.909020 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:18.909335 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:18.909410 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:18.911966 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.131:58473: BlockingRecv error: recv got EOF from 127.25.127.131:58473 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 08:24:19 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:19 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:19.642964 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:19.637924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:40515)
0619 08:24:19.638115 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:19.638127 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:19.638202 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:19.638433 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:19.638434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:19.638605 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:19.638658 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:19.638660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:19.639397 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:19.639398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:19.639943 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:19.639945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:19.640363 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:19.640363 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:19.640408 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:19.640871 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:19.640876 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:19.642859 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
W20260619 08:24:19.643025 2821 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:19.638116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:40515 (local address 127.25.127.190:36493)
0619 08:24:19.638262 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:19.638264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:19.638273 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:24:19.638302 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:19.638303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:19.638313 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:19.638388 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:19.638755 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:19.639326 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:19.640030 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:19.640296 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:19.640354 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:19.642949 (+ 2595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.129:40515: BlockingRecv error: recv got EOF from 127.25.127.129:40515 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
Jun 19 08:24:19 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:19 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:19.798599 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:19.788032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:60415)
0619 08:24:19.788248 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:19.788261 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:19.788365 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:19.788574 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:19.788575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:19.788774 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:19.788827 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:19.788829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:19.789730 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:19.789731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:19.790287 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:19.790290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:19.790717 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:19.790718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:19.790787 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:19.796251 (+ 5464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:19.796259 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:19.798533 (+ 2274us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":88,"threads_started":1}
W20260619 08:24:19.798789 2821 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:19.788174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:60415 (local address 127.25.127.190:36493)
0619 08:24:19.788245 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:19.788247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:19.788350 (+ 103us) server_negotiation.cc:407] Connection header received
0619 08:24:19.788458 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:19.788460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:19.788474 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:19.788565 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:19.788924 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:19.789584 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:19.790396 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:19.790603 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:19.790706 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:19.798675 (+ 7969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.130:60415: BlockingRecv error: recv got EOF from 127.25.127.130:60415 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Jun 19 08:24:19 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:19 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:19.917183 2823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:19.912443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.131:35699)
0619 08:24:19.912590 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:19.912601 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:19.912708 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:19.912874 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:19.912875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:19.913061 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:19.913112 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:19.913114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:19.913821 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:19.913822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:19.914360 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:19.914362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:19.914734 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:19.914735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:19.914796 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:19.915267 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:19.915272 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:19.917115 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
W20260619 08:24:19.917315 2821 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:19.912607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:35699 (local address 127.25.127.190:36493)
0619 08:24:19.912710 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:19.912712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:19.912723 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:24:19.912775 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:19.912776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:19.912796 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:19.912862 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:19.913207 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:19.913731 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:19.914455 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:19.914657 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:19.914736 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:19.917237 (+ 2501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.131:35699: BlockingRecv error: recv got EOF from 127.25.127.131:35699 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Jun 19 08:24:20 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:20 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:20.648478 2824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:20.643521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:34337)
0619 08:24:20.643675 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:20.643686 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:20.643744 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:20.643911 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:20.643911 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:20.644083 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:20.644133 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:20.644134 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:20.644900 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:20.644901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:20.645418 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:20.645419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:20.645853 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:20.645854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:20.645914 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:20.646405 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:20.646410 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:20.648404 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
W20260619 08:24:20.648572 2825 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:20.643628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:34337 (local address 127.25.127.190:36493)
0619 08:24:20.643788 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:20.643790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:20.643798 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:24:20.643824 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:20.643825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:20.643835 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:20.643896 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:20.644222 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:20.644828 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:20.645496 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:20.645779 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:20.645852 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:20.648499 (+ 2647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.129:34337: BlockingRecv error: recv got EOF from 127.25.127.129:34337 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":55,"threads_started":1}
Jun 19 08:24:20 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:20 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:20.804812 2826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:20.799069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:54825)
0619 08:24:20.799187 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:20.799203 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:20.799243 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:20.799434 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:20.799434 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:20.799601 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:20.799651 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:20.799652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:20.800707 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:20.800708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:20.801260 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:20.801261 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:20.801766 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:20.801767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:20.801810 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:20.802432 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:20.802438 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:20.804711 (+ 2273us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":65,"threads_started":1}
W20260619 08:24:20.805265 2825 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:20.799153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:54825 (local address 127.25.127.190:36493)
0619 08:24:20.799217 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:20.799220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:20.799255 (+ 35us) server_negotiation.cc:407] Connection header received
0619 08:24:20.799343 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:20.799344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:20.799358 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:20.799428 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:20.799720 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:20.800623 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:20.801351 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:20.801689 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:20.801763 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:20.805176 (+ 3413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.130:54825: BlockingRecv error: recv got EOF from 127.25.127.130:54825 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Jun 19 08:24:20 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:20 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 08:24:20.922875 2825 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:20.917928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:37001 (local address 127.25.127.190:36493)
0619 08:24:20.918017 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:20.918020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:20.918031 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:24:20.918109 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:20.918110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:20.918125 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:20.918176 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:20.918552 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:20.919119 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:20.919865 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:20.920065 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:20.920189 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:20.922815 (+ 2626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.131:37001: BlockingRecv error: recv got EOF from 127.25.127.131:37001 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 08:24:20.922868 2827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:20.917767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.131:37001)
0619 08:24:20.917937 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:20.917948 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:20.918039 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:20.918193 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:20.918194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:20.918392 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:20.918444 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:20.918446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:20.919234 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:20.919236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:20.919768 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:20.919769 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:20.920193 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:20.920195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:20.920267 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:20.920773 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:20.920778 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:20.922719 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
Jun 19 08:24:21 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:21 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:21.653719 2828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:21.649007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:35621)
0619 08:24:21.649161 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:21.649172 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:21.649237 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:21.649396 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:21.649397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:21.649578 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:21.649639 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:21.649641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:21.650325 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:21.650326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:21.650897 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:21.650899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:21.651255 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:21.651256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:21.651301 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:21.651764 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:21.651769 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:21.653607 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
W20260619 08:24:21.653790 2829 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:21.649106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:35621 (local address 127.25.127.190:36493)
0619 08:24:21.649241 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:21.649243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:21.649253 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:24:21.649280 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:21.649281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:21.649292 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:21.649354 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:21.649716 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:21.650243 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:21.650978 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:21.651172 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:21.651225 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:21.653736 (+ 2511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.129:35621: BlockingRecv error: recv got EOF from 127.25.127.129:35621 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":76,"threads_started":1}
Jun 19 08:24:21 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:21 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:21.810812 2830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:21.805486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:56959)
0619 08:24:21.805681 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:21.805693 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:21.805760 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:21.805904 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:21.805904 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:21.806078 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:21.806130 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:21.806132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:21.806914 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:21.806914 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:21.807506 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:21.807508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:21.807951 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:21.807951 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:21.808064 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:21.808676 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:21.808687 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:21.810691 (+ 2004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":99,"threads_started":1}
W20260619 08:24:21.810855 2829 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:21.805658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:56959 (local address 127.25.127.190:36493)
0619 08:24:21.805739 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:21.805741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:21.805753 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:24:21.805816 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:21.805817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:21.805829 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:21.805887 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:21.806210 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:21.806824 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:21.807611 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:21.807872 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:21.807934 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:21.810789 (+ 2855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.130:56959: BlockingRecv error: recv got EOF from 127.25.127.130:56959 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 19 08:24:21 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:21 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:21.928594 2831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:21.923407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.131:38107)
0619 08:24:21.923615 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:21.923630 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:21.923727 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:21.923915 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:21.923916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:21.924131 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:21.924248 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:21.924250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:21.924945 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:21.924947 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:21.925496 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:21.925498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:21.925905 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:21.925906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:21.925966 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:21.926525 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:21.926530 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:21.928533 (+ 2003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":96,"threads_started":1}
W20260619 08:24:21.928786 2829 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:21.923520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:38107 (local address 127.25.127.190:36493)
0619 08:24:21.923635 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:21.923637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:21.923713 (+ 76us) server_negotiation.cc:407] Connection header received
0619 08:24:21.923804 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:21.923805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:21.923822 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:21.923906 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:21.924333 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:21.924848 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:21.925613 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:21.925808 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:21.925903 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:21.928717 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.131:38107: BlockingRecv error: recv got EOF from 127.25.127.131:38107 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 19 08:24:22 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:22 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:22.659538 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:22.654320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:34787)
0619 08:24:22.654469 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:22.654481 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:22.654541 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:22.654747 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:22.654748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:22.654926 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:22.654978 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:22.654979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:22.655668 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:22.655669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:22.656243 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:22.656244 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:22.656714 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:22.656715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:22.656766 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:22.657328 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:22.657332 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:22.659469 (+ 2137us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
W20260619 08:24:22.659647 2833 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:22.654412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:34787 (local address 127.25.127.190:36493)
0619 08:24:22.654548 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:22.654555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:22.654563 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:24:22.654587 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:22.654588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:22.654598 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:22.654666 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:22.655059 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:22.655577 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:22.656410 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:22.656613 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:22.656678 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:22.659571 (+ 2893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.129:34787: BlockingRecv error: recv got EOF from 127.25.127.129:34787 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":73,"threads_started":1}
Jun 19 08:24:22 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:22 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:22.816426 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:22.811407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:43499)
0619 08:24:22.811590 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:22.811606 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:22.811676 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:22.811831 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:22.811832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:22.812000 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:22.812074 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:22.812075 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:22.812994 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:22.812995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:22.813526 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:22.813528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:22.813901 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:22.813902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:22.813945 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:22.814478 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:22.814483 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:22.816362 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":104,"threads_started":1}
W20260619 08:24:22.816557 2833 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:22.811542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:43499 (local address 127.25.127.190:36493)
0619 08:24:22.811614 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:22.811620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:22.811659 (+ 39us) server_negotiation.cc:407] Connection header received
0619 08:24:22.811740 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:22.811741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:22.811751 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:22.811811 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:22.812182 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:22.812918 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:22.813629 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:22.813824 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:22.813891 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:22.816473 (+ 2582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.130:43499: BlockingRecv error: recv got EOF from 127.25.127.130:43499 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 19 08:24:22 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:22 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:22.934027 2835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:22.929094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.131:35483)
0619 08:24:22.929232 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:22.929249 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:22.929303 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:22.929498 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:22.929499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:22.929707 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:22.929768 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:22.929770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:22.930426 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:22.930427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:22.931002 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:22.931004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:22.931336 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:22.931336 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:22.931380 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:22.931840 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:22.931845 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:22.933957 (+ 2112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
W20260619 08:24:22.934155 2833 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:22.929181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:35483 (local address 127.25.127.190:36493)
0619 08:24:22.929279 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:22.929281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:22.929293 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:24:22.929358 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:22.929359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:22.929371 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:22.929433 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:22.929848 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:22.930353 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:22.931080 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:22.931269 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:22.931321 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:22.934055 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.131:35483: BlockingRecv error: recv got EOF from 127.25.127.131:35483 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Jun 19 08:24:23 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:23 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:23.665021 2836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:23.660207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:52163)
0619 08:24:23.660365 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:23.660378 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:23.660451 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:23.660586 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:23.660587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:23.660760 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:23.660816 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:23.660818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:23.661545 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:23.661546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:23.662087 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:23.662089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:23.662463 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:23.662464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:23.662505 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:23.663013 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:23.663018 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:23.664959 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":72,"threads_started":1}
W20260619 08:24:23.665194 2837 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:23.660266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:52163 (local address 127.25.127.190:36493)
0619 08:24:23.660417 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:23.660419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:23.660432 (+ 13us) server_negotiation.cc:407] Connection header received
0619 08:24:23.660501 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:23.660502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:23.660513 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:23.660574 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:23.660925 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:23.661459 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:23.662182 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:23.662394 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:23.662454 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:23.665093 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.129:52163: BlockingRecv error: recv got EOF from 127.25.127.129:52163 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":73,"threads_started":1}
Jun 19 08:24:23 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:23 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:23.822497 2838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:23.817004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:39417)
0619 08:24:23.817294 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:23.817310 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:23.817396 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:23.817564 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:23.817565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:23.817768 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:23.817819 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:23.817821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:23.818524 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:23.818526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:23.819103 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:23.819106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:23.819483 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:23.819484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:23.819538 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:23.820092 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:23.820097 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:23.822432 (+ 2335us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":93,"threads_started":1}
W20260619 08:24:23.822634 2837 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:23.817119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:39417 (local address 127.25.127.190:36493)
0619 08:24:23.817299 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:23.817302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:23.817374 (+ 72us) server_negotiation.cc:407] Connection header received
0619 08:24:23.817463 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:23.817464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:23.817483 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:23.817546 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:23.817912 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:23.818427 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:23.819200 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:23.819404 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:23.819485 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:23.822544 (+ 3059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.130:39417: BlockingRecv error: recv got EOF from 127.25.127.130:39417 (error 108)
Metrics: {"server-negotiator.queue_time_us":149}
Jun 19 08:24:23 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:23 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.131@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:23.939109 2839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:23.934524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.131:57955)
0619 08:24:23.934675 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:23.934688 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:23.934745 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:23.934886 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:23.934887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:23.935058 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:23.935108 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:23.935109 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:23.935774 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:23.935776 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:23.936351 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:23.936353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:23.936672 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:23.936673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:23.936724 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:23.937165 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:23.937170 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:23.939046 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
W20260619 08:24:23.939245 2837 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:23.934612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:57955 (local address 127.25.127.190:36493)
0619 08:24:23.934679 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:23.934681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:23.934783 (+ 102us) server_negotiation.cc:407] Connection header received
0619 08:24:23.934807 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:23.934808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:23.934820 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:23.934862 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:23.935187 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:23.935683 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:23.936424 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:23.936605 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:23.936658 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:23.939160 (+ 2502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.131:57955: BlockingRecv error: recv got EOF from 127.25.127.131:57955 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 08:24:24 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:24 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.129@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:24.670477 2840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:24.665679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.129:44689)
0619 08:24:24.665839 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:24.665854 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:24.665925 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:24.666072 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:24.666073 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:24.666248 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:24.666299 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:24.666301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:24.666954 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:24.666955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:24.667511 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:24.667513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:24.667875 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:24.667875 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:24.667921 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:24.668573 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:24.668579 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:24.670406 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":83,"threads_started":1}
W20260619 08:24:24.670625 2841 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:24.665761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:44689 (local address 127.25.127.190:36493)
0619 08:24:24.665915 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:24.665917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:24.665926 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:24:24.665984 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:24.665985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:24.665996 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:24.666056 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:24.666362 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:24.666883 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:24.667586 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:24.667803 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:24.667860 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:24.670533 (+ 2673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.129:44689: BlockingRecv error: recv got EOF from 127.25.127.129:44689 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":74,"threads_started":1}
Jun 19 08:24:24 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:24:24 dist-test-slave-5rhg krb5kdc[1781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.127.130@KRBTEST.COM for oryx/127.25.127.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:24:24.831907 2842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:24.823059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36493 (local address 127.25.127.130:49963)
0619 08:24:24.823197 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:24.823208 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:24.823277 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:24.823444 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:24.823445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:24.823630 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:24.823746 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:24.823749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:24.824452 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:24.824453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:24.825017 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:24.825018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:24.825340 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:24.825340 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:24.825379 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:24.829702 (+ 4323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:24.829709 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:24.831828 (+ 2119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.127.190:36493: Server oryx/127.25.127.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
W20260619 08:24:24.832041 2841 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:24.823093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:49963 (local address 127.25.127.190:36493)
0619 08:24:24.823140 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:24.823143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:24.823274 (+ 131us) server_negotiation.cc:407] Connection header received
0619 08:24:24.823337 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:24.823337 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:24.823352 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:24.823420 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:24.823845 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:24.824381 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:24.825089 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:24.825276 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:24.825327 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:24.831956 (+ 6629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.127.130:49963: BlockingRecv error: recv got EOF from 127.25.127.130:49963 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:24:24.855556 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 2365
I20260619 08:24:24.860394 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 2517
I20260619 08:24:24.864807 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 2649
I20260619 08:24:24.869405 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 2295
2026-06-19T08:24:24Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17503 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T08:24:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:24:24Z Disabled control of system clock
I20260619 08:24:24.898005 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:36729
--webserver_interface=127.25.127.190
--webserver_port=0
--builtin_ntp_servers=127.25.127.148:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:36729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 08:24:24.984831 2846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:24.985085 2846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:24.985111 2846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:24.986452 2846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:24:24.986510 2846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:24.986526 2846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:24.986539 2846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:24:24.986567 2846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:24:24.988085 2846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:36729
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:36729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:24.988425 2846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:24.988782 2846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:24.994078 2851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:24.994086 2852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:24.994086 2854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:24.994172 2846 server_base.cc:1034] running on GCE node
I20260619 08:24:24.994596 2846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:24.994922 2846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:24.996091 2846 hybrid_clock.cc:648] HybridClock initialized: now 1781857464996083 us; error 37 us; skew 500 ppm
I20260619 08:24:24.997403 2846 webserver.cc:460] Webserver started at http://127.25.127.190:32989/ using document root <none> and password file <none>
I20260619 08:24:24.997689 2846 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:24.997767 2846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:24.997926 2846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:24.999063 2846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "35664409fc7e4a7c9cd4417af0c4e8f1"
format_stamp: "Formatted at 2026-06-19 08:24:24 on dist-test-slave-5rhg"
server_key: "55d3f8c41a00792f8853a57f5056759a"
server_key_iv: "67a673554caee853c2f078192f3be2ac"
server_key_version: "encryptionkey@0"
I20260619 08:24:24.999445 2846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "35664409fc7e4a7c9cd4417af0c4e8f1"
format_stamp: "Formatted at 2026-06-19 08:24:24 on dist-test-slave-5rhg"
server_key: "55d3f8c41a00792f8853a57f5056759a"
server_key_iv: "67a673554caee853c2f078192f3be2ac"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.000911 2846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:24:25.001799 2860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:25.001971 2846 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:25.002072 2846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "35664409fc7e4a7c9cd4417af0c4e8f1"
format_stamp: "Formatted at 2026-06-19 08:24:24 on dist-test-slave-5rhg"
server_key: "55d3f8c41a00792f8853a57f5056759a"
server_key_iv: "67a673554caee853c2f078192f3be2ac"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.002159 2846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:25.022430 2846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:25.022821 2846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:25.022992 2846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:25.027344 2846 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:36729
I20260619 08:24:25.027379 2912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:36729 every 8 connection(s)
I20260619 08:24:25.027762 2846 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:24:25.028518 2913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:25.031371 2913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1: Bootstrap starting.
I20260619 08:24:25.032022 2913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:25.032366 2913 log.cc:824] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1: Log is configured to *not* fsync() on all Append() calls
I20260619 08:24:25.032960 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 2846
I20260619 08:24:25.033049 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:24:25.033223 2913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1: No bootstrap required, opened a new log
I20260619 08:24:25.033340 26110 external_mini_cluster.cc:1428] Setting key 7ff9d2ee302a5305a2798f557a7c5fb0
I20260619 08:24:25.035508 2913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35664409fc7e4a7c9cd4417af0c4e8f1" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36729 } }
I20260619 08:24:25.035679 2913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:25.035758 2913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35664409fc7e4a7c9cd4417af0c4e8f1, State: Initialized, Role: FOLLOWER
I20260619 08:24:25.035912 2913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35664409fc7e4a7c9cd4417af0c4e8f1" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36729 } }
I20260619 08:24:25.036008 2913 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:24:25.036052 2913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:24:25.036103 2913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:25.036758 2913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35664409fc7e4a7c9cd4417af0c4e8f1" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36729 } }
I20260619 08:24:25.036875 2913 leader_election.cc:302] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35664409fc7e4a7c9cd4417af0c4e8f1; no voters:
I20260619 08:24:25.037052 2913 leader_election.cc:288] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:24:25.037076 2918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:24:25.037314 2913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:24:25.037317 2918 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1 [term 1 LEADER]: Becoming Leader. State: Replica: 35664409fc7e4a7c9cd4417af0c4e8f1, State: Running, Role: LEADER
I20260619 08:24:25.038148 2918 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35664409fc7e4a7c9cd4417af0c4e8f1" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36729 } }
I20260619 08:24:25.038846 2920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35664409fc7e4a7c9cd4417af0c4e8f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35664409fc7e4a7c9cd4417af0c4e8f1" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36729 } } }
I20260619 08:24:25.038970 2920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1 [sys.catalog]: This master's current role is: LEADER
I20260619 08:24:25.039469 2919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35664409fc7e4a7c9cd4417af0c4e8f1. Latest consensus state: current_term: 1 leader_uuid: "35664409fc7e4a7c9cd4417af0c4e8f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35664409fc7e4a7c9cd4417af0c4e8f1" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36729 } } }
I20260619 08:24:25.039547 2919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1 [sys.catalog]: This master's current role is: LEADER
W20260619 08:24:25.040088 2932 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:24:25.040159 2932 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:24:25.040278 2934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:24:25.040475 2934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:24:25.040839 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:25.033924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40702 (local address 127.25.127.190:36729)
0619 08:24:25.034157 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:25.034170 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:25.034196 (+ 26us) server_negotiation.cc:407] Connection header received
0619 08:24:25.034640 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:25.034646 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:25.034711 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:25.034828 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:25.035075 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:25.036612 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:25.037718 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:25.038466 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:25.038604 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:25.039832 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:25.039835 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:25.039835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:24:25.040043 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:25.040089 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:24:25.040106 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:25.040480 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:25.040514 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 08:24:25.040622 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":112,"threads_started":1}
I20260619 08:24:25.042287 2934 catalog_manager.cc:1347] Generated new cluster ID: c619322c6a1a472cbdaf7fb4680355aa
I20260619 08:24:25.042343 2934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:24:25.076375 2934 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:24:25.077121 2934 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:24:25.085009 2934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35664409fc7e4a7c9cd4417af0c4e8f1: Generated new TSK 0
I20260619 08:24:25.085245 2934 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:24:25.095016 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:36729
--builtin_ntp_servers=127.25.127.148:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 08:24:25.180975 2937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:25.181226 2937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:25.181267 2937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:25.182474 2937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:25.182541 2937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:25.182586 2937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:24:25.183960 2937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:36729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:25.184258 2937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:25.184574 2937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:25.189536 2943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:25.189571 2942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:25.189536 2945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:25.189976 2937 server_base.cc:1034] running on GCE node
I20260619 08:24:25.190377 2937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:25.190666 2937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:25.191854 2937 hybrid_clock.cc:648] HybridClock initialized: now 1781857465191842 us; error 35 us; skew 500 ppm
I20260619 08:24:25.193370 2937 webserver.cc:460] Webserver started at http://127.25.127.129:44387/ using document root <none> and password file <none>
I20260619 08:24:25.193661 2937 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:25.193742 2937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:25.193893 2937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:25.195041 2937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "a65aa760fd05412ba2b85e66d883f8f3"
format_stamp: "Formatted at 2026-06-19 08:24:25 on dist-test-slave-5rhg"
server_key: "0725335d4fec16a72cdb53eb921ee049"
server_key_iv: "9762ac4a8bb1ca3e4a3e8ba3d20810f3"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.195441 2937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "a65aa760fd05412ba2b85e66d883f8f3"
format_stamp: "Formatted at 2026-06-19 08:24:25 on dist-test-slave-5rhg"
server_key: "0725335d4fec16a72cdb53eb921ee049"
server_key_iv: "9762ac4a8bb1ca3e4a3e8ba3d20810f3"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.196907 2937 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:24:25.197764 2951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:25.197940 2937 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:24:25.198024 2937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "a65aa760fd05412ba2b85e66d883f8f3"
format_stamp: "Formatted at 2026-06-19 08:24:25 on dist-test-slave-5rhg"
server_key: "0725335d4fec16a72cdb53eb921ee049"
server_key_iv: "9762ac4a8bb1ca3e4a3e8ba3d20810f3"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.198099 2937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:25.215126 2937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:25.215410 2937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:25.215512 2937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:25.215705 2937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:24:25.216068 2937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:24:25.216109 2937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:25.216231 2937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:24:25.216260 2937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:25.222579 2937 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:32855
I20260619 08:24:25.222633 3064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:32855 every 8 connection(s)
I20260619 08:24:25.222975 2937 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:24:25.228652 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:25.223806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:60217 (local address 127.25.127.190:36729)
0619 08:24:25.223892 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:25.223894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:25.224124 (+ 230us) server_negotiation.cc:407] Connection header received
0619 08:24:25.224674 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:25.224675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:25.224692 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:25.224761 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:25.225476 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:25.225998 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:25.227127 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:25.227336 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:25.227395 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:25.228396 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:25.228397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:25.228398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:24:25.228403 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:25.228418 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:24:25.228422 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:25.228475 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:25.228559 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 08:24:25.228589 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 08:24:25.228916 3067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:25.223778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36729 (local address 127.25.127.129:60217)
0619 08:24:25.223991 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:25.224034 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:25.224575 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:25.224846 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:25.224850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:25.224860 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:25.225396 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:25.225401 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:25.226180 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:25.226181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:25.227037 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:25.227039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:25.227426 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:25.227427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:25.227549 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:25.228251 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:24:25.228251 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:25.228283 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:24:25.228307 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:25.228488 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:25.228493 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:25.228495 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:24:25.228550 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 08:24:25.228635 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
I20260619 08:24:25.229506 3065 heartbeater.cc:347] Connected to a master server at 127.25.127.190:36729
I20260619 08:24:25.229607 3065 heartbeater.cc:464] Registering TS with master...
I20260619 08:24:25.229768 3065 heartbeater.cc:511] Master 127.25.127.190:36729 requested a full tablet report, sending...
I20260619 08:24:25.230321 2877 ts_manager.cc:194] Registered new tserver with Master: a65aa760fd05412ba2b85e66d883f8f3 (127.25.127.129:32855)
I20260619 08:24:25.230670 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 2937
I20260619 08:24:25.230751 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:24:25.230955 26110 external_mini_cluster.cc:1428] Setting key 2d0f197765c63c8d06f179c1b834ca63
I20260619 08:24:25.231118 2877 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.127.129:60217
I20260619 08:24:25.232055 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:36729
--builtin_ntp_servers=127.25.127.148:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 08:24:25.315712 3068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:25.315939 3068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:25.315963 3068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:25.317354 3068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:25.317409 3068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:25.317436 3068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:24:25.318769 3068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:36729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:25.318976 3068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:25.319247 3068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:25.324549 3073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:25.324558 3074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:25.324559 3076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:25.324609 3068 server_base.cc:1034] running on GCE node
I20260619 08:24:25.325030 3068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:25.325320 3068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:25.326484 3068 hybrid_clock.cc:648] HybridClock initialized: now 1781857465326467 us; error 28 us; skew 500 ppm
I20260619 08:24:25.327886 3068 webserver.cc:460] Webserver started at http://127.25.127.130:46619/ using document root <none> and password file <none>
I20260619 08:24:25.328204 3068 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:25.328269 3068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:25.328451 3068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:25.329579 3068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "9a3ee73bee634d898b92b3cf0368d9f4"
format_stamp: "Formatted at 2026-06-19 08:24:25 on dist-test-slave-5rhg"
server_key: "61a1da8d0deeaab95857f45b05e73c9b"
server_key_iv: "4245550ee75409c52c874270e71b407f"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.329967 3068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "9a3ee73bee634d898b92b3cf0368d9f4"
format_stamp: "Formatted at 2026-06-19 08:24:25 on dist-test-slave-5rhg"
server_key: "61a1da8d0deeaab95857f45b05e73c9b"
server_key_iv: "4245550ee75409c52c874270e71b407f"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.331349 3068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:24:25.332134 3082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:25.332351 3068 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:25.332448 3068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "9a3ee73bee634d898b92b3cf0368d9f4"
format_stamp: "Formatted at 2026-06-19 08:24:25 on dist-test-slave-5rhg"
server_key: "61a1da8d0deeaab95857f45b05e73c9b"
server_key_iv: "4245550ee75409c52c874270e71b407f"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.332533 3068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:25.359628 3068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:25.360082 3068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:25.360299 3068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:25.360563 3068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:24:25.360945 3068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:24:25.361004 3068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:25.361047 3068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:24:25.361078 3068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:25.367044 3068 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:45939
I20260619 08:24:25.367077 3195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:45939 every 8 connection(s)
I20260619 08:24:25.367465 3068 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:24:25.372859 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:25.368301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:33937 (local address 127.25.127.190:36729)
0619 08:24:25.368348 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:25.368350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:25.368623 (+ 273us) server_negotiation.cc:407] Connection header received
0619 08:24:25.369159 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:25.369160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:25.369172 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:25.369237 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:25.369923 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:25.370438 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:25.371428 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:25.371610 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:25.371662 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:25.372638 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:25.372639 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:25.372639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:24:25.372643 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:25.372656 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:24:25.372660 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:25.372708 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:25.372777 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 08:24:25.372797 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:24:25.373075 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:25.368338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36729 (local address 127.25.127.130:33937)
0619 08:24:25.368494 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:25.368528 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:25.369038 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:25.369290 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:25.369293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:25.369301 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:25.369823 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:25.369826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:25.370525 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:25.370526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:25.371350 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:25.371352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:25.371683 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:25.371684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:25.371824 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:25.372499 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:24:25.372500 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:25.372532 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:24:25.372557 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:25.372711 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:25.372714 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:25.372716 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:24:25.372763 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 08:24:25.372845 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":65,"threads_started":1}
I20260619 08:24:25.373639 3196 heartbeater.cc:347] Connected to a master server at 127.25.127.190:36729
I20260619 08:24:25.373754 3196 heartbeater.cc:464] Registering TS with master...
I20260619 08:24:25.373941 3196 heartbeater.cc:511] Master 127.25.127.190:36729 requested a full tablet report, sending...
I20260619 08:24:25.374392 2876 ts_manager.cc:194] Registered new tserver with Master: 9a3ee73bee634d898b92b3cf0368d9f4 (127.25.127.130:45939)
I20260619 08:24:25.374778 2876 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.127.130:33937
I20260619 08:24:25.377301 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 3068
I20260619 08:24:25.377382 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:24:25.377569 26110 external_mini_cluster.cc:1428] Setting key 4b8bf0a727c48093727dde712fcd16b1
I20260619 08:24:25.378566 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:36729
--builtin_ntp_servers=127.25.127.148:35319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 08:24:25.462877 3199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:25.463101 3199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:25.463126 3199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:25.464459 3199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:25.464505 3199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:25.464531 3199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:24:25.465915 3199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:35319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:36729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:25.466190 3199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:25.466514 3199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:25.471920 3204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:25.471863 3207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:25.472076 3199 server_base.cc:1034] running on GCE node
W20260619 08:24:25.471863 3205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:25.472390 3199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:25.472662 3199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:25.473820 3199 hybrid_clock.cc:648] HybridClock initialized: now 1781857465473797 us; error 43 us; skew 500 ppm
I20260619 08:24:25.475219 3199 webserver.cc:460] Webserver started at http://127.25.127.131:38323/ using document root <none> and password file <none>
I20260619 08:24:25.475503 3199 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:25.475553 3199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:25.475718 3199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:25.476969 3199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "a586376bfc65480c83bf3bca9b3d0561"
format_stamp: "Formatted at 2026-06-19 08:24:25 on dist-test-slave-5rhg"
server_key: "c33a2d5a9949edb89a41850fb71faac6"
server_key_iv: "bb75e3909ce34ddf2f8444886584f914"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.477334 3199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "a586376bfc65480c83bf3bca9b3d0561"
format_stamp: "Formatted at 2026-06-19 08:24:25 on dist-test-slave-5rhg"
server_key: "c33a2d5a9949edb89a41850fb71faac6"
server_key_iv: "bb75e3909ce34ddf2f8444886584f914"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.478693 3199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:24:25.479485 3213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:25.479707 3199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:25.479807 3199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "a586376bfc65480c83bf3bca9b3d0561"
format_stamp: "Formatted at 2026-06-19 08:24:25 on dist-test-slave-5rhg"
server_key: "c33a2d5a9949edb89a41850fb71faac6"
server_key_iv: "bb75e3909ce34ddf2f8444886584f914"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.479869 3199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:25.499742 3199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:25.500075 3199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:25.500279 3199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:25.500574 3199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:24:25.500905 3199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:24:25.500936 3199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:25.500958 3199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:24:25.500972 3199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:25.507490 3199 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:38265
I20260619 08:24:25.507575 3326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:38265 every 8 connection(s)
I20260619 08:24:25.507952 3199 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:24:25.513145 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:25.508800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:56167 (local address 127.25.127.190:36729)
0619 08:24:25.508871 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:25.508873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:25.509103 (+ 230us) server_negotiation.cc:407] Connection header received
0619 08:24:25.509605 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:25.509606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:25.509616 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:25.509679 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:25.510382 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:25.510872 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:25.511863 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:25.512044 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:25.512097 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:25.512917 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:25.512918 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:25.512918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:24:25.512923 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:25.512935 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:24:25.512939 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:25.512982 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:25.513061 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 08:24:25.513083 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:24:25.513402 3329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:25.508756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:36729 (local address 127.25.127.131:56167)
0619 08:24:25.508966 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:25.509010 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:25.509519 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:25.509740 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:25.509743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:25.509752 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:25.510284 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:25.510300 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:25.510960 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:25.510961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:25.511785 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:25.511788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:25.512122 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:25.512123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:25.512264 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:25.512787 (+ 523us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:24:25.512788 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:25.512816 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:24:25.512840 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:25.512994 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:25.512998 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:25.513000 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:24:25.513052 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:24:25.513136 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":102,"threads_started":1}
I20260619 08:24:25.513597 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 3199
I20260619 08:24:25.513692 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:24:25.513891 26110 external_mini_cluster.cc:1428] Setting key e9100770b363c792b06baf259d3580ec
I20260619 08:24:25.513934 3327 heartbeater.cc:347] Connected to a master server at 127.25.127.190:36729
I20260619 08:24:25.514032 3327 heartbeater.cc:464] Registering TS with master...
I20260619 08:24:25.514233 3327 heartbeater.cc:511] Master 127.25.127.190:36729 requested a full tablet report, sending...
I20260619 08:24:25.514657 2877 ts_manager.cc:194] Registered new tserver with Master: a586376bfc65480c83bf3bca9b3d0561 (127.25.127.131:38265)
I20260619 08:24:25.515058 2877 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.127.131:56167
I20260619 08:24:25.525839 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 08:24:25.527952 2917 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:25.527304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40714 (local address 127.25.127.190:36729)
0619 08:24:25.527398 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:25.527401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:25.527513 (+ 112us) server_negotiation.cc:407] Connection header received
0619 08:24:25.527610 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:25.527612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:25.527622 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:25.527711 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:25.527865 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40714: BlockingRecv error: recv got EOF from 127.0.0.1:40714 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 08:24:25.528003 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 2937
I20260619 08:24:25.532783 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 3068
I20260619 08:24:25.537833 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 3199
I20260619 08:24:25.542230 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 2846
2026-06-19T08:24:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (671 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T08:24:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:24:25Z Disabled control of system clock
I20260619 08:24:25.569094 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:41929
--webserver_interface=127.25.127.190
--webserver_port=0
--builtin_ntp_servers=127.25.127.148:42839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:41929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 08:24:25.651486 3338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:25.651721 3338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:25.651744 3338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:25.652899 3338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:24:25.652937 3338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:25.652951 3338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:25.652963 3338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:24:25.652976 3338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:24:25.654369 3338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:41929
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:41929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:25.654579 3338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:25.654836 3338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:25.659699 3343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:25.659699 3346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:25.659898 3338 server_base.cc:1034] running on GCE node
W20260619 08:24:25.660087 3344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:25.660391 3338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:25.660708 3338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:25.661885 3338 hybrid_clock.cc:648] HybridClock initialized: now 1781857465661870 us; error 32 us; skew 500 ppm
I20260619 08:24:25.663272 3338 webserver.cc:460] Webserver started at http://127.25.127.190:42259/ using document root <none> and password file <none>
I20260619 08:24:25.663540 3338 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:25.663615 3338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:25.663784 3338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:25.665021 3338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "5c81305c9a2c4f69b15246255bbfa695"
format_stamp: "Formatted at 2026-06-19 08:24:25 on dist-test-slave-5rhg"
server_key: "6daf3b2872be8150e399058813f434e4"
server_key_iv: "4e5a69b9e73758abd588a1bb8bb21f8d"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.665406 3338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "5c81305c9a2c4f69b15246255bbfa695"
format_stamp: "Formatted at 2026-06-19 08:24:25 on dist-test-slave-5rhg"
server_key: "6daf3b2872be8150e399058813f434e4"
server_key_iv: "4e5a69b9e73758abd588a1bb8bb21f8d"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.666790 3338 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:24:25.667636 3352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:25.667814 3338 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:24:25.667927 3338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "5c81305c9a2c4f69b15246255bbfa695"
format_stamp: "Formatted at 2026-06-19 08:24:25 on dist-test-slave-5rhg"
server_key: "6daf3b2872be8150e399058813f434e4"
server_key_iv: "4e5a69b9e73758abd588a1bb8bb21f8d"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.668005 3338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:25.671383 3338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:25.671679 3338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:25.671836 3338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:25.675690 3338 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:41929
I20260619 08:24:25.675725 3404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:41929 every 8 connection(s)
I20260619 08:24:25.676074 3338 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:24:25.676805 3405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:25.679615 3405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695: Bootstrap starting.
I20260619 08:24:25.680297 3405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:25.680608 3405 log.cc:824] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695: Log is configured to *not* fsync() on all Append() calls
I20260619 08:24:25.681354 3405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695: No bootstrap required, opened a new log
I20260619 08:24:25.682976 3405 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c81305c9a2c4f69b15246255bbfa695" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41929 } }
I20260619 08:24:25.683122 3405 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:25.683145 3405 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c81305c9a2c4f69b15246255bbfa695, State: Initialized, Role: FOLLOWER
I20260619 08:24:25.683314 3405 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c81305c9a2c4f69b15246255bbfa695" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41929 } }
I20260619 08:24:25.683389 3405 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:24:25.683411 3405 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:24:25.683440 3405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:25.683732 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 3338
I20260619 08:24:25.683848 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:24:25.684038 26110 external_mini_cluster.cc:1428] Setting key 478511025894ab7ac9b32fa239de1ece
I20260619 08:24:25.684208 3405 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c81305c9a2c4f69b15246255bbfa695" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41929 } }
I20260619 08:24:25.684334 3405 leader_election.cc:302] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c81305c9a2c4f69b15246255bbfa695; no voters:
I20260619 08:24:25.684543 3405 leader_election.cc:288] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:24:25.684607 3408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:24:25.684814 3408 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695 [term 1 LEADER]: Becoming Leader. State: Replica: 5c81305c9a2c4f69b15246255bbfa695, State: Running, Role: LEADER
I20260619 08:24:25.684912 3408 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c81305c9a2c4f69b15246255bbfa695" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41929 } }
I20260619 08:24:25.684967 3405 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:24:25.685266 3408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c81305c9a2c4f69b15246255bbfa695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c81305c9a2c4f69b15246255bbfa695" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41929 } } }
I20260619 08:24:25.685371 3408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695 [sys.catalog]: This master's current role is: LEADER
I20260619 08:24:25.685546 3411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c81305c9a2c4f69b15246255bbfa695. Latest consensus state: current_term: 1 leader_uuid: "5c81305c9a2c4f69b15246255bbfa695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c81305c9a2c4f69b15246255bbfa695" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41929 } } }
I20260619 08:24:25.685642 3411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695 [sys.catalog]: This master's current role is: LEADER
I20260619 08:24:25.686345 3415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:24:25.686969 3410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:25.684626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39564 (local address 127.25.127.190:41929)
0619 08:24:25.684905 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:25.684915 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:25.684952 (+ 37us) server_negotiation.cc:407] Connection header received
0619 08:24:25.685078 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:25.685085 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:25.685133 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:25.685839 (+ 706us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:25.686518 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:25.686523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:25.686523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:24:25.686528 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:25.686574 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:24:25.686588 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:25.686653 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:25.686720 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 08:24:25.686791 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":90,"threads_started":1}
I20260619 08:24:25.687070 3415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:24:25.688689 3415 catalog_manager.cc:1347] Generated new cluster ID: 1ef0d2b626a746859dc5b6c3876ed820
I20260619 08:24:25.688751 3415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:24:25.718904 3415 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:24:25.719141 3415 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:24:25.724782 3415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c81305c9a2c4f69b15246255bbfa695: Generated new TSK 0
I20260619 08:24:25.725023 3415 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:24:25.733263 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:41929
--builtin_ntp_servers=127.25.127.148:42839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 08:24:25.824823 3429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:25.825094 3429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:25.825135 3429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:25.826416 3429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:25.826488 3429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:25.826534 3429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:24:25.827971 3429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:41929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:25.828346 3429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:25.828670 3429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:25.834329 3435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:25.834355 3437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:25.834355 3434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:25.834672 3429 server_base.cc:1034] running on GCE node
I20260619 08:24:25.834933 3429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:25.835166 3429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:25.836303 3429 hybrid_clock.cc:648] HybridClock initialized: now 1781857465836287 us; error 31 us; skew 500 ppm
I20260619 08:24:25.837641 3429 webserver.cc:460] Webserver started at http://127.25.127.129:36901/ using document root <none> and password file <none>
I20260619 08:24:25.837913 3429 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:25.837993 3429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:25.838160 3429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:25.839339 3429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "b1d0aa0b3b184b56a87c9b6bcb015754"
format_stamp: "Formatted at 2026-06-19 08:24:25 on dist-test-slave-5rhg"
server_key: "9249fd3ffda22c290bbfcedff30b5219"
server_key_iv: "39c89f8c420b98bea0bdc5d5971df2cb"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.839715 3429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "b1d0aa0b3b184b56a87c9b6bcb015754"
format_stamp: "Formatted at 2026-06-19 08:24:25 on dist-test-slave-5rhg"
server_key: "9249fd3ffda22c290bbfcedff30b5219"
server_key_iv: "39c89f8c420b98bea0bdc5d5971df2cb"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.841148 3429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:24:25.841892 3443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:25.842078 3429 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:24:25.842187 3429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "b1d0aa0b3b184b56a87c9b6bcb015754"
format_stamp: "Formatted at 2026-06-19 08:24:25 on dist-test-slave-5rhg"
server_key: "9249fd3ffda22c290bbfcedff30b5219"
server_key_iv: "39c89f8c420b98bea0bdc5d5971df2cb"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.842264 3429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:25.845818 3429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:25.846112 3429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:25.846261 3429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:25.846472 3429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:24:25.846791 3429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:24:25.846843 3429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:25.846884 3429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:24:25.846910 3429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:25.853406 3429 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:34961
I20260619 08:24:25.853447 3556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:34961 every 8 connection(s)
I20260619 08:24:25.853863 3429 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:24:25.857252 3410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:25.854601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:52993 (local address 127.25.127.190:41929)
0619 08:24:25.854665 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:25.854667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:25.854952 (+ 285us) server_negotiation.cc:407] Connection header received
0619 08:24:25.855501 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:25.855503 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:25.855512 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:25.855570 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:25.856944 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:25.856946 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:25.856946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:24:25.856951 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:25.856969 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:24:25.856973 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:25.857055 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:25.857160 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 08:24:25.857179 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:24:25.857455 3559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:25.854600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:41929 (local address 127.25.127.129:52993)
0619 08:24:25.854784 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:25.854848 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:25.855387 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:25.855649 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:25.855654 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:25.855664 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:25.856783 (+ 1119us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:24:25.856784 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:25.856815 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:24:25.856846 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:25.857054 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:25.857061 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:25.857064 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:24:25.857159 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 08:24:25.857234 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260619 08:24:25.858011 3557 heartbeater.cc:347] Connected to a master server at 127.25.127.190:41929
I20260619 08:24:25.858136 3557 heartbeater.cc:464] Registering TS with master...
I20260619 08:24:25.858307 3557 heartbeater.cc:511] Master 127.25.127.190:41929 requested a full tablet report, sending...
I20260619 08:24:25.858903 3369 ts_manager.cc:194] Registered new tserver with Master: b1d0aa0b3b184b56a87c9b6bcb015754 (127.25.127.129:34961)
I20260619 08:24:25.858912 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 3429
I20260619 08:24:25.859012 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:24:25.859261 26110 external_mini_cluster.cc:1428] Setting key b863d715d78806032195e4f5d9217833
I20260619 08:24:25.860321 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:41929
--builtin_ntp_servers=127.25.127.148:42839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 08:24:25.945823 3560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:25.946121 3560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:25.946147 3560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:25.947358 3560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:25.947397 3560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:25.947424 3560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:24:25.948843 3560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:41929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:25.949071 3560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:25.949340 3560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:25.954483 3566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:25.954523 3565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:25.954527 3560 server_base.cc:1034] running on GCE node
W20260619 08:24:25.954483 3568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:25.954982 3560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:25.955251 3560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:25.956425 3560 hybrid_clock.cc:648] HybridClock initialized: now 1781857465956404 us; error 39 us; skew 500 ppm
I20260619 08:24:25.957839 3560 webserver.cc:460] Webserver started at http://127.25.127.130:40553/ using document root <none> and password file <none>
I20260619 08:24:25.958098 3560 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:25.958184 3560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:25.958339 3560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:25.959506 3560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "66fa12165f5640e199d18361ca58c848"
format_stamp: "Formatted at 2026-06-19 08:24:25 on dist-test-slave-5rhg"
server_key: "5015ea6f1a6d10f403dd2ca4dbc01379"
server_key_iv: "4a471e0c6719f28baa8e1c3c8398822f"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.959896 3560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "66fa12165f5640e199d18361ca58c848"
format_stamp: "Formatted at 2026-06-19 08:24:25 on dist-test-slave-5rhg"
server_key: "5015ea6f1a6d10f403dd2ca4dbc01379"
server_key_iv: "4a471e0c6719f28baa8e1c3c8398822f"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.961393 3560 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:24:25.962235 3574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:25.962414 3560 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:25.962517 3560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "66fa12165f5640e199d18361ca58c848"
format_stamp: "Formatted at 2026-06-19 08:24:25 on dist-test-slave-5rhg"
server_key: "5015ea6f1a6d10f403dd2ca4dbc01379"
server_key_iv: "4a471e0c6719f28baa8e1c3c8398822f"
server_key_version: "encryptionkey@0"
I20260619 08:24:25.962611 3560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:25.966289 3560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:25.966588 3560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:25.966742 3560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:25.966954 3560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:24:25.967288 3560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:24:25.967348 3560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:25.967389 3560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:24:25.967418 3560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:25.973635 3560 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:36009
I20260619 08:24:25.973712 3687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:36009 every 8 connection(s)
I20260619 08:24:25.974064 3560 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:24:25.975310 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 3560
I20260619 08:24:25.975423 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:24:25.975612 26110 external_mini_cluster.cc:1428] Setting key 7a3fc04530473ade29f7068ef1ea3953
I20260619 08:24:25.976629 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:41929
--builtin_ntp_servers=127.25.127.148:42839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 08:24:25.977777 3410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:25.974804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:42119 (local address 127.25.127.190:41929)
0619 08:24:25.974868 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:25.974870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:25.975126 (+ 256us) server_negotiation.cc:407] Connection header received
0619 08:24:25.975767 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:25.975769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:25.975776 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:25.975838 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:25.977487 (+ 1649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:25.977489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:25.977489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:24:25.977494 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:25.977513 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:24:25.977518 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:25.977588 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:25.977704 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 08:24:25.977719 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 08:24:25.978045 3690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:25.974802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:41929 (local address 127.25.127.130:42119)
0619 08:24:25.975002 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:25.975036 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:25.975636 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:25.975930 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:25.975935 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:25.975947 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:25.977214 (+ 1267us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:24:25.977215 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:25.977261 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:24:25.977304 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:25.977601 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:25.977608 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:25.977611 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:24:25.977697 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 08:24:25.977779 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260619 08:24:25.978536 3688 heartbeater.cc:347] Connected to a master server at 127.25.127.190:41929
I20260619 08:24:25.978709 3688 heartbeater.cc:464] Registering TS with master...
I20260619 08:24:25.978885 3688 heartbeater.cc:511] Master 127.25.127.190:41929 requested a full tablet report, sending...
I20260619 08:24:25.979281 3369 ts_manager.cc:194] Registered new tserver with Master: 66fa12165f5640e199d18361ca58c848 (127.25.127.130:36009)
W20260619 08:24:26.065778 3691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:26.066008 3691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:26.066033 3691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:26.067391 3691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:26.067443 3691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:26.067474 3691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:24:26.069095 3691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:41929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:26.069393 3691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:26.069737 3691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:26.075136 3697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:26.075306 3691 server_base.cc:1034] running on GCE node
W20260619 08:24:26.075147 3696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:26.075383 3699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:26.075644 3691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:26.075894 3691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:26.077107 3691 hybrid_clock.cc:648] HybridClock initialized: now 1781857466077092 us; error 43 us; skew 500 ppm
I20260619 08:24:26.078624 3691 webserver.cc:460] Webserver started at http://127.25.127.131:39563/ using document root <none> and password file <none>
I20260619 08:24:26.078895 3691 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:26.078950 3691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:26.079144 3691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:26.080458 3691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "6ffbd15615594b30acb8838f86f5046e"
format_stamp: "Formatted at 2026-06-19 08:24:26 on dist-test-slave-5rhg"
server_key: "68a3d84b41105072164d0e21993022df"
server_key_iv: "661715cbbe3b09e402a52faeeaeb4013"
server_key_version: "encryptionkey@0"
I20260619 08:24:26.080873 3691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "6ffbd15615594b30acb8838f86f5046e"
format_stamp: "Formatted at 2026-06-19 08:24:26 on dist-test-slave-5rhg"
server_key: "68a3d84b41105072164d0e21993022df"
server_key_iv: "661715cbbe3b09e402a52faeeaeb4013"
server_key_version: "encryptionkey@0"
I20260619 08:24:26.082324 3691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:24:26.083217 3705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:26.083406 3691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:26.083494 3691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "6ffbd15615594b30acb8838f86f5046e"
format_stamp: "Formatted at 2026-06-19 08:24:26 on dist-test-slave-5rhg"
server_key: "68a3d84b41105072164d0e21993022df"
server_key_iv: "661715cbbe3b09e402a52faeeaeb4013"
server_key_version: "encryptionkey@0"
I20260619 08:24:26.083556 3691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:26.087220 3691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:26.087462 3691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:26.087560 3691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:26.087718 3691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:24:26.087978 3691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:24:26.088009 3691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:26.088030 3691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:24:26.088042 3691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:26.093943 3691 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:41639
I20260619 08:24:26.094027 3818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:41639 every 8 connection(s)
I20260619 08:24:26.094295 3691 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:24:26.097560 3410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:26.095159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:59361 (local address 127.25.127.190:41929)
0619 08:24:26.095244 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:26.095246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:26.095499 (+ 253us) server_negotiation.cc:407] Connection header received
0619 08:24:26.095991 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:26.095992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:26.095998 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:26.096059 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:26.097340 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:26.097341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:26.097342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:24:26.097346 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:26.097359 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:24:26.097363 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:26.097414 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:26.097487 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 08:24:26.097501 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 08:24:26.097813 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:26.095121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:41929 (local address 127.25.127.131:59361)
0619 08:24:26.095373 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:26.095410 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:26.095880 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:26.096183 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:26.096188 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:26.096198 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:26.097177 (+ 979us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:24:26.097177 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:26.097208 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:24:26.097239 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:26.097428 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:26.097433 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:26.097435 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:24:26.097495 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 08:24:26.097547 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":123,"threads_started":1}
I20260619 08:24:26.098326 3819 heartbeater.cc:347] Connected to a master server at 127.25.127.190:41929
I20260619 08:24:26.098428 3819 heartbeater.cc:464] Registering TS with master...
I20260619 08:24:26.098585 3819 heartbeater.cc:511] Master 127.25.127.190:41929 requested a full tablet report, sending...
I20260619 08:24:26.098973 3369 ts_manager.cc:194] Registered new tserver with Master: 6ffbd15615594b30acb8838f86f5046e (127.25.127.131:41639)
I20260619 08:24:26.102309 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 3691
I20260619 08:24:26.102392 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:24:26.102604 26110 external_mini_cluster.cc:1428] Setting key 4289f2616b3a7a583c67240bb31a08f5
I20260619 08:24:26.103539 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 08:24:26.105554 3410 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:24:26.105101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39568 (local address 127.25.127.190:41929)
0619 08:24:26.105147 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:26.105148 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:26.105263 (+ 115us) server_negotiation.cc:407] Connection header received
0619 08:24:26.105320 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:26.105321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:26.105325 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:26.105370 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:26.105480 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39568: BlockingRecv error: recv got EOF from 127.0.0.1:39568 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:24:26.105669 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 3429
I20260619 08:24:26.111202 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 3560
I20260619 08:24:26.115527 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 3691
I20260619 08:24:26.119776 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 3338
2026-06-19T08:24:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (578 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:24:26 dist-test-slave-5rhg krb5kdc[3829](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:24:26 dist-test-slave-5rhg krb5kdc[3829](info): set up 2 sockets
Jun 19 08:24:26 dist-test-slave-5rhg krb5kdc[3829](info): commencing operation
krb5kdc: starting...
W20260619 08:24:27.170241 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:24:27 dist-test-slave-5rhg krb5kdc[3829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:24:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:24:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:24:27.296550 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:38873
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:39919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:38873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:24:27.380040 3842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:27.380331 3842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:27.380357 3842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:27.381693 3842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:24:27.381744 3842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:27.381759 3842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:27.381772 3842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:24:27.381784 3842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:24:27.383353 3842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:39919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:38873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:38873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:27.383646 3842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:27.383886 3842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:27.389053 3847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:27.389024 3848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:27.389021 3850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:27.389339 3842 server_base.cc:1034] running on GCE node
I20260619 08:24:27.389585 3842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:27.389904 3842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:27.391103 3842 hybrid_clock.cc:648] HybridClock initialized: now 1781857467391067 us; error 44 us; skew 500 ppm
Jun 19 08:24:27 dist-test-slave-5rhg krb5kdc[3829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857467, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:24:27.393908 3842 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:24:27.394387 3842 webserver.cc:460] Webserver started at http://127.25.127.190:38567/ using document root <none> and password file <none>
I20260619 08:24:27.394624 3842 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:27.394675 3842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:27.394809 3842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:27.395932 3842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "3b4439b192354b19aadb1eb76b5c331f"
format_stamp: "Formatted at 2026-06-19 08:24:27 on dist-test-slave-5rhg"
server_key: "c7911087cf6483534b6821765974e615"
server_key_iv: "70de6230071ac548fde8865c1934fee8"
server_key_version: "encryptionkey@0"
I20260619 08:24:27.396340 3842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "3b4439b192354b19aadb1eb76b5c331f"
format_stamp: "Formatted at 2026-06-19 08:24:27 on dist-test-slave-5rhg"
server_key: "c7911087cf6483534b6821765974e615"
server_key_iv: "70de6230071ac548fde8865c1934fee8"
server_key_version: "encryptionkey@0"
I20260619 08:24:27.397723 3842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:24:27.398530 3857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:27.398725 3842 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:24:27.398823 3842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "3b4439b192354b19aadb1eb76b5c331f"
format_stamp: "Formatted at 2026-06-19 08:24:27 on dist-test-slave-5rhg"
server_key: "c7911087cf6483534b6821765974e615"
server_key_iv: "70de6230071ac548fde8865c1934fee8"
server_key_version: "encryptionkey@0"
I20260619 08:24:27.398901 3842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:27.413305 3842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:27.415839 3842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:27.416023 3842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:27.420094 3842 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:38873
I20260619 08:24:27.420127 3909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:38873 every 8 connection(s)
I20260619 08:24:27.420588 3842 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:24:27.421578 3910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:27.421921 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 3842
I20260619 08:24:27.422017 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:24:27.422214 26110 external_mini_cluster.cc:1428] Setting key edbb3aade54ea97961420b5c735ecc3f
I20260619 08:24:27.424804 3910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f: Bootstrap starting.
I20260619 08:24:27.425443 3910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:27.425740 3910 log.cc:824] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f: Log is configured to *not* fsync() on all Append() calls
I20260619 08:24:27.426450 3910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f: No bootstrap required, opened a new log
I20260619 08:24:27.428342 3910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4439b192354b19aadb1eb76b5c331f" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38873 } }
I20260619 08:24:27.428506 3910 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:27.428555 3910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b4439b192354b19aadb1eb76b5c331f, State: Initialized, Role: FOLLOWER
I20260619 08:24:27.428719 3910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4439b192354b19aadb1eb76b5c331f" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38873 } }
I20260619 08:24:27.428831 3910 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:24:27.428876 3910 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Jun 19 08:24:27 dist-test-slave-5rhg krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:24:27.428928 3910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:27.429705 3910 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4439b192354b19aadb1eb76b5c331f" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38873 } }
I20260619 08:24:27.429833 3910 leader_election.cc:302] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b4439b192354b19aadb1eb76b5c331f; no voters:
I20260619 08:24:27.430025 3910 leader_election.cc:288] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:24:27.430151 3915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:24:27.430343 3910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:24:27.430356 3915 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f [term 1 LEADER]: Becoming Leader. State: Replica: 3b4439b192354b19aadb1eb76b5c331f, State: Running, Role: LEADER
I20260619 08:24:27.430559 3915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4439b192354b19aadb1eb76b5c331f" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38873 } }
I20260619 08:24:27.432060 3916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b4439b192354b19aadb1eb76b5c331f. Latest consensus state: current_term: 1 leader_uuid: "3b4439b192354b19aadb1eb76b5c331f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4439b192354b19aadb1eb76b5c331f" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38873 } } }
I20260619 08:24:27.432175 3916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f [sys.catalog]: This master's current role is: LEADER
W20260619 08:24:27.432192 3930 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:24:27.432238 3930 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:24:27.432348 3932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:24:27.432473 3917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b4439b192354b19aadb1eb76b5c331f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4439b192354b19aadb1eb76b5c331f" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 38873 } } }
I20260619 08:24:27.432581 3917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f [sys.catalog]: This master's current role is: LEADER
I20260619 08:24:27.432515 3932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:24:27.434528 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:27.422784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42980 (local address 127.25.127.190:38873)
0619 08:24:27.422956 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:27.422965 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:27.422990 (+ 25us) server_negotiation.cc:407] Connection header received
0619 08:24:27.423431 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:27.423437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:27.423483 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:27.423603 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:27.424510 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:27.425501 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:27.426324 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:27.426799 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:27.426922 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:27.429498 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:27.429502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:27.429503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:27.429513 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:27.432632 (+ 3119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:27.433079 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:27.433083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:27.433083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:27.433201 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:27.433475 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:27.433476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:27.433476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:27.433798 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:27.433954 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:27.434168 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 08:24:27.434322 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260619 08:24:27.434643 3932 catalog_manager.cc:1347] Generated new cluster ID: f98c1eccfe5f4769b49edc673aac7d7d
I20260619 08:24:27.434746 3932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:24:27.438956 3932 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:24:27.439656 3932 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:24:27.447772 3932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f: Generated new TSK 0
I20260619 08:24:27.448064 3932 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
I20260619 08:24:27.486814 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:38873
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:39919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:24:27.579753 3936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:27.579977 3936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:27.580003 3936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:27.581259 3936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:27.581308 3936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:27.581338 3936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:24:27.582921 3936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:39919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:38873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:27.583184 3936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:27.583455 3936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:27.588773 3941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:27.588975 3936 server_base.cc:1034] running on GCE node
W20260619 08:24:27.588774 3944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:27.588774 3942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:27.589308 3936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:27.589587 3936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:27.590761 3936 hybrid_clock.cc:648] HybridClock initialized: now 1781857467590747 us; error 32 us; skew 500 ppm
Jun 19 08:24:27 dist-test-slave-5rhg krb5kdc[3829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857467, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:24:27.593564 3936 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:24:27.594018 3936 webserver.cc:460] Webserver started at http://127.25.127.129:34025/ using document root <none> and password file <none>
I20260619 08:24:27.594267 3936 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:27.594312 3936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:27.594478 3936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:27.595620 3936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "5a53c492a7a7409e9717fe2b3da68f9a"
format_stamp: "Formatted at 2026-06-19 08:24:27 on dist-test-slave-5rhg"
server_key: "48a935dfa7b3b92fed8f7bf9090843b2"
server_key_iv: "93585a5247cb5c44d65e252aa5aa4b60"
server_key_version: "encryptionkey@0"
I20260619 08:24:27.595984 3936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "5a53c492a7a7409e9717fe2b3da68f9a"
format_stamp: "Formatted at 2026-06-19 08:24:27 on dist-test-slave-5rhg"
server_key: "48a935dfa7b3b92fed8f7bf9090843b2"
server_key_iv: "93585a5247cb5c44d65e252aa5aa4b60"
server_key_version: "encryptionkey@0"
I20260619 08:24:27.597334 3936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:24:27.598073 3951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:27.598254 3936 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:24:27.598341 3936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "5a53c492a7a7409e9717fe2b3da68f9a"
format_stamp: "Formatted at 2026-06-19 08:24:27 on dist-test-slave-5rhg"
server_key: "48a935dfa7b3b92fed8f7bf9090843b2"
server_key_iv: "93585a5247cb5c44d65e252aa5aa4b60"
server_key_version: "encryptionkey@0"
I20260619 08:24:27.598415 3936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:27.618927 3936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:27.621068 3936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:27.621227 3936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:27.621503 3936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:24:27.621833 3936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:24:27.621868 3936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:27.621914 3936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:24:27.621945 3936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:27.627969 3936 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:37179
I20260619 08:24:27.628013 4064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:37179 every 8 connection(s)
I20260619 08:24:27.628419 3936 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:24:27.632975 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 3936
I20260619 08:24:27.633102 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:24:27.633314 26110 external_mini_cluster.cc:1428] Setting key 62831ff58d999305c7a551d323226998
Jun 19 08:24:27 dist-test-slave-5rhg krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857467, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:24:27.639038 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:27.629172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:41309 (local address 127.25.127.190:38873)
0619 08:24:27.629252 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:27.629254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:27.629483 (+ 229us) server_negotiation.cc:407] Connection header received
0619 08:24:27.629991 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:27.629992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:27.630002 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:27.630066 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:27.631214 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:27.631781 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:27.632933 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:27.633188 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:27.633263 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:27.635970 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:27.635972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:27.635973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:27.635980 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:27.637527 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:27.637989 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:27.637991 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:27.637991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:27.638058 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:27.638337 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:27.638338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:27.638338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:27.638462 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:27.638535 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:27.638845 (+ 310us) server_negotiation.cc:299] Negotiation successful
0619 08:24:27.638941 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 08:24:27.639266 4067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:27.629159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:38873 (local address 127.25.127.129:41309)
0619 08:24:27.629339 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:27.629384 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:27.629902 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:27.630206 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:27.630211 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:27.630886 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:27.631126 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:27.631131 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:27.631859 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:27.631859 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:27.632837 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:27.632841 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:27.633291 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:27.633292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:27.633450 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:27.634164 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:27.634176 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:27.635865 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:27.637638 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:27.637647 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:27.637649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:27.637906 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:27.638129 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:27.638130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:27.638130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:27.638260 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:27.638541 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:27.638549 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:27.638706 (+ 157us) client_negotiation.cc:769] Sending connection context
0619 08:24:27.638826 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 08:24:27.639007 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":107,"threads_started":1}
I20260619 08:24:27.639923 4065 heartbeater.cc:347] Connected to a master server at 127.25.127.190:38873
I20260619 08:24:27.640019 4065 heartbeater.cc:464] Registering TS with master...
I20260619 08:24:27.640290 4065 heartbeater.cc:511] Master 127.25.127.190:38873 requested a full tablet report, sending...
I20260619 08:24:27.640877 3874 ts_manager.cc:194] Registered new tserver with Master: 5a53c492a7a7409e9717fe2b3da68f9a (127.25.127.129:37179)
I20260619 08:24:27.641506 3874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:41309
WARNING: no policy specified for kudu/127.25.127.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
I20260619 08:24:27.659175 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:38873
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:39919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:24:27.742663 4070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:27.742878 4070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:27.742902 4070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:27.744095 4070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:27.744160 4070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:27.744190 4070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:24:27.745587 4070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:39919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:38873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:27.745781 4070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:27.745954 4070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:27.751205 4075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:27.751204 4076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:27.751199 4078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:27.751492 4070 server_base.cc:1034] running on GCE node
I20260619 08:24:27.751799 4070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:27.752087 4070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:27.753234 4070 hybrid_clock.cc:648] HybridClock initialized: now 1781857467753215 us; error 39 us; skew 500 ppm
Jun 19 08:24:27 dist-test-slave-5rhg krb5kdc[3829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857467, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:24:27.755860 4070 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:24:27.756511 4070 webserver.cc:460] Webserver started at http://127.25.127.130:41423/ using document root <none> and password file <none>
I20260619 08:24:27.756757 4070 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:27.756826 4070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:27.756971 4070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:27.758306 4070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "6607a81b601041e0b92aff2204324f50"
format_stamp: "Formatted at 2026-06-19 08:24:27 on dist-test-slave-5rhg"
server_key: "879e23db8f29a6a80b438dec961ee735"
server_key_iv: "89b7871eee3c6dcb8db91ec911346a38"
server_key_version: "encryptionkey@0"
I20260619 08:24:27.758744 4070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "6607a81b601041e0b92aff2204324f50"
format_stamp: "Formatted at 2026-06-19 08:24:27 on dist-test-slave-5rhg"
server_key: "879e23db8f29a6a80b438dec961ee735"
server_key_iv: "89b7871eee3c6dcb8db91ec911346a38"
server_key_version: "encryptionkey@0"
I20260619 08:24:27.760447 4070 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260619 08:24:27.761198 4085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:27.761391 4070 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:24:27.761466 4070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "6607a81b601041e0b92aff2204324f50"
format_stamp: "Formatted at 2026-06-19 08:24:27 on dist-test-slave-5rhg"
server_key: "879e23db8f29a6a80b438dec961ee735"
server_key_iv: "89b7871eee3c6dcb8db91ec911346a38"
server_key_version: "encryptionkey@0"
I20260619 08:24:27.761519 4070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:27.808000 4070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:27.810556 4070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:27.810702 4070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:27.810909 4070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:24:27.811249 4070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:24:27.811286 4070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:27.811332 4070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:24:27.811359 4070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:27.817373 4070 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:39927
I20260619 08:24:27.817417 4198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:39927 every 8 connection(s)
I20260619 08:24:27.817766 4070 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:24:27 dist-test-slave-5rhg krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857467, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:24:27.825043 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 4070
I20260619 08:24:27.825127 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:24:27.825310 26110 external_mini_cluster.cc:1428] Setting key adb409f1a5038c822169a7c6bc34cd1f
I20260619 08:24:27.828266 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:27.818567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:54707 (local address 127.25.127.190:38873)
0619 08:24:27.818640 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:27.818643 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:27.818878 (+ 235us) server_negotiation.cc:407] Connection header received
0619 08:24:27.819430 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:27.819431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:27.819443 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:27.819497 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:27.820572 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:27.821094 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:27.822132 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:27.822342 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:27.822427 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:27.824717 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:27.824719 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:27.824719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:27.824726 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:27.826709 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:27.827108 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:27.827109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:27.827109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:27.827177 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:27.827413 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:27.827414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:27.827415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:27.827581 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:27.827662 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:27.828035 (+ 373us) server_negotiation.cc:299] Negotiation successful
0619 08:24:27.828112 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:24:27.828495 4201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:27.818544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:38873 (local address 127.25.127.130:54707)
0619 08:24:27.818743 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:27.818782 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:27.819318 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:27.819588 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:27.819592 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:27.820256 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:27.820465 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:27.820471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:27.821193 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:27.821194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:27.822028 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:27.822029 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:27.822437 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:27.822438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:27.822575 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:27.823076 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:27.823085 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:27.824624 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:27.826802 (+ 2178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:27.826808 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:27.826811 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:27.827034 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:27.827243 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:27.827244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:27.827244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:27.827348 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:27.827662 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:27.827669 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:27.827856 (+ 187us) client_negotiation.cc:769] Sending connection context
0619 08:24:27.827989 (+ 133us) client_negotiation.cc:240] Negotiation successful
0619 08:24:27.828218 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":114,"threads_started":1}
I20260619 08:24:27.829008 4199 heartbeater.cc:347] Connected to a master server at 127.25.127.190:38873
I20260619 08:24:27.829141 4199 heartbeater.cc:464] Registering TS with master...
I20260619 08:24:27.829327 4199 heartbeater.cc:511] Master 127.25.127.190:38873 requested a full tablet report, sending...
I20260619 08:24:27.829754 3874 ts_manager.cc:194] Registered new tserver with Master: 6607a81b601041e0b92aff2204324f50 (127.25.127.130:39927)
I20260619 08:24:27.830128 3874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:54707
WARNING: no policy specified for kudu/127.25.127.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
I20260619 08:24:27.851240 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:38873
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:39919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:24:27.932920 4204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:24:27.933173 4204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:24:27.933215 4204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:24:27.934423 4204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:24:27.934492 4204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:24:27.934540 4204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:24:27.936618 4204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:39919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:38873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:24:27.936909 4204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:24:27.937152 4204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:24:27.942579 4209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:27.942679 4212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:24:27.942586 4210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:24:27.942623 4204 server_base.cc:1034] running on GCE node
I20260619 08:24:27.942929 4204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:24:27.943239 4204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:24:27.944415 4204 hybrid_clock.cc:648] HybridClock initialized: now 1781857467944409 us; error 38 us; skew 500 ppm
Jun 19 08:24:27 dist-test-slave-5rhg krb5kdc[3829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857467, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:24:27.946990 4204 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:24:27.947499 4204 webserver.cc:460] Webserver started at http://127.25.127.131:38807/ using document root <none> and password file <none>
I20260619 08:24:27.947819 4204 fs_manager.cc:359] Metadata directory not provided
I20260619 08:24:27.947896 4204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:24:27.948061 4204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:24:27.949187 4204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "91053153d0e442c182c7e21c02b1d3c5"
format_stamp: "Formatted at 2026-06-19 08:24:27 on dist-test-slave-5rhg"
server_key: "bd3100c721940f54287a1c1169970ebb"
server_key_iv: "77c075e98d8f98aa466b7d25241db541"
server_key_version: "encryptionkey@0"
I20260619 08:24:27.949559 4204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "91053153d0e442c182c7e21c02b1d3c5"
format_stamp: "Formatted at 2026-06-19 08:24:27 on dist-test-slave-5rhg"
server_key: "bd3100c721940f54287a1c1169970ebb"
server_key_iv: "77c075e98d8f98aa466b7d25241db541"
server_key_version: "encryptionkey@0"
I20260619 08:24:27.950918 4204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:24:27.951790 4219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:27.951999 4204 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:24:27.952118 4204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "91053153d0e442c182c7e21c02b1d3c5"
format_stamp: "Formatted at 2026-06-19 08:24:27 on dist-test-slave-5rhg"
server_key: "bd3100c721940f54287a1c1169970ebb"
server_key_iv: "77c075e98d8f98aa466b7d25241db541"
server_key_version: "encryptionkey@0"
I20260619 08:24:27.952237 4204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:24:27.969238 4204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:24:27.971201 4204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:24:27.971374 4204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:24:27.971621 4204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:24:27.971953 4204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:24:27.972008 4204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:27.972050 4204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:24:27.972092 4204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:24:27.978259 4204 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:35023
I20260619 08:24:27.978304 4332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:35023 every 8 connection(s)
I20260619 08:24:27.978644 4204 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:24:27 dist-test-slave-5rhg krb5kdc[3829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857467, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:24:27.986284 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 4204
I20260619 08:24:27.986388 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:24:27.986582 26110 external_mini_cluster.cc:1428] Setting key 971b2aed0bbe257e0250363b43bd2491
I20260619 08:24:27.988811 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:27.979367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:55723 (local address 127.25.127.190:38873)
0619 08:24:27.979420 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:27.979421 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:27.979645 (+ 224us) server_negotiation.cc:407] Connection header received
0619 08:24:27.980284 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:27.980286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:27.980300 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:27.980387 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:27.981607 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:27.982126 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:27.983172 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:27.983379 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:27.983467 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:27.985693 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:27.985695 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:27.985695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:27.985704 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:27.987414 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:27.987857 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:27.987858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:27.987858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:27.987912 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:27.988223 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:27.988224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:27.988225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:27.988359 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:27.988425 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:27.988677 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 08:24:27.988745 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:24:27.988971 4335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:27.979342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:38873 (local address 127.25.127.131:55723)
0619 08:24:27.979502 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:27.979553 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:27.980136 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:27.980458 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:27.980463 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:27.981281 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:24:27.981487 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:27.981492 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:27.982221 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:27.982222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:27.983079 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:27.983081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:27.983478 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:27.983479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:27.983616 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:27.984119 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:24:27.984127 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:24:27.985583 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:24:27.987525 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:27.987532 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:27.987534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:27.987792 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:27.987972 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:24:27.987973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:24:27.987973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:24:27.988070 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:24:27.988425 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:24:27.988430 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:24:27.988553 (+ 123us) client_negotiation.cc:769] Sending connection context
0619 08:24:27.988640 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 08:24:27.988767 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":70,"threads_started":1}
I20260619 08:24:27.989470 4333 heartbeater.cc:347] Connected to a master server at 127.25.127.190:38873
I20260619 08:24:27.989588 4333 heartbeater.cc:464] Registering TS with master...
I20260619 08:24:27.989802 4333 heartbeater.cc:511] Master 127.25.127.190:38873 requested a full tablet report, sending...
I20260619 08:24:27.990159 3874 ts_manager.cc:194] Registered new tserver with Master: 91053153d0e442c182c7e21c02b1d3c5 (127.25.127.131:35023)
I20260619 08:24:27.990501 3874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:55723
I20260619 08:24:27.998559 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:24:28.006217 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:27.999819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54248 (local address 127.25.127.190:38873)
0619 08:24:27.999909 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:27.999912 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:28.000000 (+ 88us) server_negotiation.cc:407] Connection header received
0619 08:24:28.000079 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:28.000080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:28.000090 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:28.000191 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:24:28.000923 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.001430 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.002260 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.002457 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.002535 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:28.003748 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:24:28.003750 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:24:28.003750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:24:28.003758 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:24:28.004993 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:28.005344 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:28.005345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:28.005345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:28.005403 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:24:28.005607 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:24:28.005608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:24:28.005608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:24:28.005766 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:24:28.005875 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:28.006008 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 08:24:28.006096 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 08:24:28.007795 3874 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:24:28.009555 3874 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:24:28.016502 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:28.012318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53492 (local address 127.25.127.130:39927)
0619 08:24:28.012504 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:28.012510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:28.012736 (+ 226us) server_negotiation.cc:407] Connection header received
0619 08:24:28.012858 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:28.012860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:28.012874 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:28.012925 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:28.013149 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.013855 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.015405 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.016100 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.016191 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:28.016240 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:28.016372 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 08:24:28.016418 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":115,"threads_started":1}
I20260619 08:24:28.016520 4345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:28.011993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:37179 (local address 127.0.0.1:51204)
0619 08:24:28.012350 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:28.012367 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:28.012410 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:28.012960 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:28.012961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:28.012962 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:28.013019 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:28.013020 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.014261 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.014262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:28.015219 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:28.015222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.016116 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.016117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:28.016249 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:28.016267 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 08:24:28.016347 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 08:24:28.016413 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260619 08:24:28.016769 4346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:28.012113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51204 (local address 127.25.127.129:37179)
0619 08:24:28.012718 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:28.012722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:28.012733 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:24:28.012775 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:28.012778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:28.012793 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:28.012870 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:28.013106 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.014180 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.015308 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.016031 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.016126 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:28.016216 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:28.016625 (+ 409us) server_negotiation.cc:299] Negotiation successful
0619 08:24:28.016672 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":116,"threads_started":1}
I20260619 08:24:28.016511 4349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:28.012514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:39927 (local address 127.0.0.1:53492)
0619 08:24:28.012654 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:28.012674 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:28.012766 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:28.013024 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:28.013024 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:28.013027 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:28.013060 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:28.013061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.013958 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.013959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:28.015326 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:28.015328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.016195 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.016196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:28.016249 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:28.016267 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 08:24:28.016347 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 08:24:28.016413 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":73,"thread_start_us":105,"threads_started":1}
I20260619 08:24:28.017529 3999 tablet_service.cc:1467] Processing CreateTablet for tablet b896965d8df941fca6d3b1259dc90c1c (DEFAULT_TABLE table=test-table [id=e5f870b7bf1c49da9304c65a4a68ce5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:24:28.017881 3999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b896965d8df941fca6d3b1259dc90c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:28.017838 4133 tablet_service.cc:1467] Processing CreateTablet for tablet b896965d8df941fca6d3b1259dc90c1c (DEFAULT_TABLE table=test-table [id=e5f870b7bf1c49da9304c65a4a68ce5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:24:28.018138 4133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b896965d8df941fca6d3b1259dc90c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:28.020829 4351 tablet_bootstrap.cc:492] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50: Bootstrap starting.
I20260619 08:24:28.020773 4350 tablet_bootstrap.cc:492] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a: Bootstrap starting.
I20260619 08:24:28.021570 4351 tablet_bootstrap.cc:654] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:28.021589 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:28.011936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:35023 (local address 127.0.0.1:43666)
0619 08:24:28.012126 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:28.012191 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:28.012261 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:28.012715 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:28.012720 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:28.012722 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:28.012800 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:28.012802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.014218 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.014219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:28.015337 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:28.015339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.021291 (+ 5952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.021292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:28.021363 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:28.021368 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:24:28.021437 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 08:24:28.021495 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":49,"thread_start_us":86,"threads_started":1}
I20260619 08:24:28.021612 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:28.012178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43666 (local address 127.25.127.131:35023)
0619 08:24:28.012501 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:28.012506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:28.012514 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:24:28.012548 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:28.012550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:28.012586 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:28.012886 (+ 300us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:28.012961 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.014101 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.016281 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.021190 (+ 4909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.021281 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:28.021342 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:28.021454 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 08:24:28.021512 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":87,"threads_started":1}
I20260619 08:24:28.021628 4350 tablet_bootstrap.cc:654] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:28.022303 4350 log.cc:824] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:24:28.022469 4351 log.cc:824] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50: Log is configured to *not* fsync() on all Append() calls
I20260619 08:24:28.022380 4267 tablet_service.cc:1467] Processing CreateTablet for tablet b896965d8df941fca6d3b1259dc90c1c (DEFAULT_TABLE table=test-table [id=e5f870b7bf1c49da9304c65a4a68ce5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:24:28.022701 4267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b896965d8df941fca6d3b1259dc90c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:24:28.023125 4350 tablet_bootstrap.cc:492] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a: No bootstrap required, opened a new log
I20260619 08:24:28.023180 4350 ts_tablet_manager.cc:1397] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:24:28.024582 4351 tablet_bootstrap.cc:492] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50: No bootstrap required, opened a new log
I20260619 08:24:28.024708 4351 ts_tablet_manager.cc:1397] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 08:24:28.024940 4350 raft_consensus.cc:348] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91053153d0e442c182c7e21c02b1d3c5" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35023 } } peers { permanent_uuid: "6607a81b601041e0b92aff2204324f50" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 39927 } } peers { permanent_uuid: "5a53c492a7a7409e9717fe2b3da68f9a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 37179 } }
I20260619 08:24:28.025094 4350 raft_consensus.cc:374] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:28.025147 4350 raft_consensus.cc:729] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a53c492a7a7409e9717fe2b3da68f9a, State: Initialized, Role: FOLLOWER
I20260619 08:24:28.025252 4350 consensus_queue.cc:260] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91053153d0e442c182c7e21c02b1d3c5" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35023 } } peers { permanent_uuid: "6607a81b601041e0b92aff2204324f50" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 39927 } } peers { permanent_uuid: "5a53c492a7a7409e9717fe2b3da68f9a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 37179 } }
I20260619 08:24:28.025527 4350 ts_tablet_manager.cc:1428] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:24:28.025723 4065 heartbeater.cc:503] Master 127.25.127.190:38873 was elected leader, sending a full tablet report...
I20260619 08:24:28.026681 4351 raft_consensus.cc:348] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91053153d0e442c182c7e21c02b1d3c5" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35023 } } peers { permanent_uuid: "6607a81b601041e0b92aff2204324f50" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 39927 } } peers { permanent_uuid: "5a53c492a7a7409e9717fe2b3da68f9a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 37179 } }
I20260619 08:24:28.026865 4351 raft_consensus.cc:374] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:28.026912 4351 raft_consensus.cc:729] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6607a81b601041e0b92aff2204324f50, State: Initialized, Role: FOLLOWER
I20260619 08:24:28.027031 4351 consensus_queue.cc:260] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91053153d0e442c182c7e21c02b1d3c5" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35023 } } peers { permanent_uuid: "6607a81b601041e0b92aff2204324f50" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 39927 } } peers { permanent_uuid: "5a53c492a7a7409e9717fe2b3da68f9a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 37179 } }
I20260619 08:24:28.027289 4351 ts_tablet_manager.cc:1428] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 08:24:28.027319 4199 heartbeater.cc:503] Master 127.25.127.190:38873 was elected leader, sending a full tablet report...
I20260619 08:24:28.027525 4355 tablet_bootstrap.cc:492] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5: Bootstrap starting.
I20260619 08:24:28.028123 4355 tablet_bootstrap.cc:654] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5: Neither blocks nor log segments found. Creating new log.
I20260619 08:24:28.028620 4355 log.cc:824] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5: Log is configured to *not* fsync() on all Append() calls
I20260619 08:24:28.029281 4355 tablet_bootstrap.cc:492] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5: No bootstrap required, opened a new log
I20260619 08:24:28.029364 4355 ts_tablet_manager.cc:1397] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:24:28.031003 4355 raft_consensus.cc:348] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91053153d0e442c182c7e21c02b1d3c5" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35023 } } peers { permanent_uuid: "6607a81b601041e0b92aff2204324f50" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 39927 } } peers { permanent_uuid: "5a53c492a7a7409e9717fe2b3da68f9a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 37179 } }
I20260619 08:24:28.031148 4355 raft_consensus.cc:374] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:24:28.031188 4355 raft_consensus.cc:729] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91053153d0e442c182c7e21c02b1d3c5, State: Initialized, Role: FOLLOWER
I20260619 08:24:28.031342 4355 consensus_queue.cc:260] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91053153d0e442c182c7e21c02b1d3c5" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35023 } } peers { permanent_uuid: "6607a81b601041e0b92aff2204324f50" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 39927 } } peers { permanent_uuid: "5a53c492a7a7409e9717fe2b3da68f9a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 37179 } }
I20260619 08:24:28.031596 4355 ts_tablet_manager.cc:1428] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:24:28.031643 4333 heartbeater.cc:503] Master 127.25.127.190:38873 was elected leader, sending a full tablet report...
W20260619 08:24:28.068320 4200 tablet.cc:2369] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:24:28.121608 4354 raft_consensus.cc:482] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:24:28.121742 4354 raft_consensus.cc:504] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91053153d0e442c182c7e21c02b1d3c5" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35023 } } peers { permanent_uuid: "6607a81b601041e0b92aff2204324f50" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 39927 } } peers { permanent_uuid: "5a53c492a7a7409e9717fe2b3da68f9a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 37179 } }
I20260619 08:24:28.122192 4354 leader_election.cc:288] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91053153d0e442c182c7e21c02b1d3c5 (127.25.127.131:35023), 6607a81b601041e0b92aff2204324f50 (127.25.127.130:39927)
I20260619 08:24:28.126240 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:28.122582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:55511 (local address 127.25.127.131:35023)
0619 08:24:28.122669 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:28.122670 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:28.122679 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:24:28.122726 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:28.122728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:28.122738 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:28.122822 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:28.123011 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.123767 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.125270 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.125859 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.125936 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:28.125944 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:28.126121 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 08:24:28.126165 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 08:24:28.126240 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:28.122670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:56475 (local address 127.25.127.130:39927)
0619 08:24:28.122745 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:28.122747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:28.122756 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:24:28.122803 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:28.122804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:28.122816 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:28.122871 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:24:28.123011 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.123760 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.125269 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.125859 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.125937 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:28.125944 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:28.126127 (+ 183us) server_negotiation.cc:299] Negotiation successful
0619 08:24:28.126165 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 08:24:28.126240 4067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:28.122367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:35023 (local address 127.25.127.129:55511)
0619 08:24:28.122430 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:28.122443 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:28.122491 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:28.122838 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:28.122839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:28.122841 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:28.122921 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:28.122923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.123849 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.123850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:28.125173 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:28.125175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.125955 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.125956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:28.126037 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:28.126038 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:24:28.126097 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 08:24:28.126150 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":46}
I20260619 08:24:28.126247 4359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:28.122521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:39927 (local address 127.25.127.129:56475)
0619 08:24:28.122687 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:28.122698 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:24:28.122753 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:24:28.122877 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:24:28.122877 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:24:28.122878 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:24:28.122934 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:28.122935 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.123849 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.123850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:28.125172 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:24:28.125175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.125955 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.125956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:24:28.126033 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:28.126038 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:24:28.126097 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 08:24:28.126150 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":70,"thread_start_us":87,"threads_started":1}
I20260619 08:24:28.126649 4153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b896965d8df941fca6d3b1259dc90c1c" candidate_uuid: "5a53c492a7a7409e9717fe2b3da68f9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6607a81b601041e0b92aff2204324f50" is_pre_election: true
I20260619 08:24:28.126649 4287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b896965d8df941fca6d3b1259dc90c1c" candidate_uuid: "5a53c492a7a7409e9717fe2b3da68f9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91053153d0e442c182c7e21c02b1d3c5" is_pre_election: true
I20260619 08:24:28.126819 4287 raft_consensus.cc:2454] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a53c492a7a7409e9717fe2b3da68f9a in term 0.
I20260619 08:24:28.126819 4153 raft_consensus.cc:2454] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a53c492a7a7409e9717fe2b3da68f9a in term 0.
I20260619 08:24:28.127060 3953 leader_election.cc:302] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a53c492a7a7409e9717fe2b3da68f9a, 6607a81b601041e0b92aff2204324f50; no voters:
I20260619 08:24:28.127223 4354 raft_consensus.cc:2781] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:24:28.127341 4354 raft_consensus.cc:482] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:24:28.127372 4354 raft_consensus.cc:3037] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:28.128259 4354 raft_consensus.cc:504] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91053153d0e442c182c7e21c02b1d3c5" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35023 } } peers { permanent_uuid: "6607a81b601041e0b92aff2204324f50" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 39927 } } peers { permanent_uuid: "5a53c492a7a7409e9717fe2b3da68f9a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 37179 } }
I20260619 08:24:28.128403 4354 leader_election.cc:288] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [CANDIDATE]: Term 1 election: Requested vote from peers 91053153d0e442c182c7e21c02b1d3c5 (127.25.127.131:35023), 6607a81b601041e0b92aff2204324f50 (127.25.127.130:39927)
I20260619 08:24:28.128603 4287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b896965d8df941fca6d3b1259dc90c1c" candidate_uuid: "5a53c492a7a7409e9717fe2b3da68f9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91053153d0e442c182c7e21c02b1d3c5"
I20260619 08:24:28.128604 4153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b896965d8df941fca6d3b1259dc90c1c" candidate_uuid: "5a53c492a7a7409e9717fe2b3da68f9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6607a81b601041e0b92aff2204324f50"
I20260619 08:24:28.128705 4287 raft_consensus.cc:3037] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:24:28.128700 4153 raft_consensus.cc:3037] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50 [term 0 FOLLOWER]: Advancing to term 1
W20260619 08:24:28.129092 4066 tablet.cc:2369] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:24:28.129549 4287 raft_consensus.cc:2454] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a53c492a7a7409e9717fe2b3da68f9a in term 1.
I20260619 08:24:28.129587 4153 raft_consensus.cc:2454] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a53c492a7a7409e9717fe2b3da68f9a in term 1.
I20260619 08:24:28.129740 3953 leader_election.cc:302] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a53c492a7a7409e9717fe2b3da68f9a, 91053153d0e442c182c7e21c02b1d3c5; no voters:
I20260619 08:24:28.129876 4354 raft_consensus.cc:2781] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:24:28.130019 4354 raft_consensus.cc:686] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [term 1 LEADER]: Becoming Leader. State: Replica: 5a53c492a7a7409e9717fe2b3da68f9a, State: Running, Role: LEADER
I20260619 08:24:28.130111 4354 consensus_queue.cc:237] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91053153d0e442c182c7e21c02b1d3c5" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35023 } } peers { permanent_uuid: "6607a81b601041e0b92aff2204324f50" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 39927 } } peers { permanent_uuid: "5a53c492a7a7409e9717fe2b3da68f9a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 37179 } }
I20260619 08:24:28.130959 3873 catalog_manager.cc:5495] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a53c492a7a7409e9717fe2b3da68f9a (127.25.127.129). New cstate: current_term: 1 leader_uuid: "5a53c492a7a7409e9717fe2b3da68f9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91053153d0e442c182c7e21c02b1d3c5" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6607a81b601041e0b92aff2204324f50" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 39927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a53c492a7a7409e9717fe2b3da68f9a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 37179 } health_report { overall_health: HEALTHY } } }
I20260619 08:24:28.161334 4346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:28.158737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.25.127.129:37179)
0619 08:24:28.158797 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:24:28.158800 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:24:28.158810 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:24:28.158862 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:24:28.158863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:24:28.158884 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:24:28.158965 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:24:28.159119 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.159600 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.160479 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:24:28.160685 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:24:28.160763 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:24:28.160958 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:24:28.161070 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:24:28.161170 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 08:24:28.161189 (+ 19us) server_negotiation.cc:299] Negotiation successful
0619 08:24:28.161210 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 08:24:28.163276 4153 raft_consensus.cc:1261] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50 [term 1 FOLLOWER]: Refusing update from remote peer 5a53c492a7a7409e9717fe2b3da68f9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:24:28.163276 4287 raft_consensus.cc:1261] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5 [term 1 FOLLOWER]: Refusing update from remote peer 5a53c492a7a7409e9717fe2b3da68f9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:24:28.163672 4360 consensus_queue.cc:1035] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6607a81b601041e0b92aff2204324f50" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 39927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:24:28.163834 4354 consensus_queue.cc:1035] T b896965d8df941fca6d3b1259dc90c1c P 5a53c492a7a7409e9717fe2b3da68f9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "91053153d0e442c182c7e21c02b1d3c5" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 35023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:24:28.166549 4363 mvcc.cc:205] Tried to move back new op lower bound from 7298488189593538560 to 7298488189461659648. Current Snapshot: MvccSnapshot[applied={T|T < 7298488189593538560}]
I20260619 08:24:28.167311 4364 mvcc.cc:205] Tried to move back new op lower bound from 7298488189593538560 to 7298488189461659648. Current Snapshot: MvccSnapshot[applied={T|T < 7298488189593538560}]
I20260619 08:24:28.168740 3874 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:24:28.168828 3874 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:24:28.169591 3874 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b4439b192354b19aadb1eb76b5c331f: Sending DeleteTablet for 3 replicas of tablet b896965d8df941fca6d3b1259dc90c1c
I20260619 08:24:28.169911 4267 tablet_service.cc:1514] Processing DeleteTablet for tablet b896965d8df941fca6d3b1259dc90c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:28 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:43666
I20260619 08:24:28.169915 3999 tablet_service.cc:1514] Processing DeleteTablet for tablet b896965d8df941fca6d3b1259dc90c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:28 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:51204
I20260619 08:24:28.170138 4133 tablet_service.cc:1514] Processing DeleteTablet for tablet b896965d8df941fca6d3b1259dc90c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:24:28 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:53492
I20260619 08:24:28.170315 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 3936
I20260619 08:24:28.170372 4375 tablet_replica.cc:330] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50: stopping tablet replica
I20260619 08:24:28.170467 4375 raft_consensus.cc:2229] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:24:28.170562 4375 raft_consensus.cc:2258] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:24:28.171557 4375 ts_tablet_manager.cc:1905] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:28.173549 4375 ts_tablet_manager.cc:1918] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:24:28.173628 4375 log.cc:1196] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/b896965d8df941fca6d3b1259dc90c1c
I20260619 08:24:28.173904 4375 ts_tablet_manager.cc:1939] T b896965d8df941fca6d3b1259dc90c1c P 6607a81b601041e0b92aff2204324f50: Deleting consensus metadata
I20260619 08:24:28.174369 3859 catalog_manager.cc:4841] TS 6607a81b601041e0b92aff2204324f50 (127.25.127.130:39927): tablet b896965d8df941fca6d3b1259dc90c1c (table test-table [id=e5f870b7bf1c49da9304c65a4a68ce5c]) successfully deleted
I20260619 08:24:28.176332 4374 tablet_replica.cc:330] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5: stopping tablet replica
I20260619 08:24:28.176487 4374 raft_consensus.cc:2229] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:24:28.176657 4374 raft_consensus.cc:2258] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 08:24:28.176921 3858 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.127.129:37179 (error 108)
I20260619 08:24:28.177178 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 4070
I20260619 08:24:28.178295 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:24:28.178169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:37179 (local address 127.0.0.1:51220)
0619 08:24:28.178227 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 08:24:28.178265 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.129:37179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 08:24:28.178416 3858 catalog_manager.cc:4584] TS 5a53c492a7a7409e9717fe2b3da68f9a (127.25.127.129:37179): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b896965d8df941fca6d3b1259dc90c1c: Network error: Client connection negotiation failed: client connection to 127.25.127.129:37179: connect: Connection refused (error 111)
I20260619 08:24:28.178460 4374 ts_tablet_manager.cc:1905] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:24:28.180475 4374 ts_tablet_manager.cc:1918] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:24:28.180564 4374 log.cc:1196] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/b896965d8df941fca6d3b1259dc90c1c
I20260619 08:24:28.180846 4374 ts_tablet_manager.cc:1939] T b896965d8df941fca6d3b1259dc90c1c P 91053153d0e442c182c7e21c02b1d3c5: Deleting consensus metadata
I20260619 08:24:28.181288 3859 catalog_manager.cc:4841] TS 91053153d0e442c182c7e21c02b1d3c5 (127.25.127.131:35023): tablet b896965d8df941fca6d3b1259dc90c1c (table test-table [id=e5f870b7bf1c49da9304c65a4a68ce5c]) successfully deleted
I20260619 08:24:28.183809 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 4204
I20260619 08:24:28.188377 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 3842
2026-06-19T08:24:28Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2068 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:24:28 dist-test-slave-5rhg krb5kdc[4379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:24:28 dist-test-slave-5rhg krb5kdc[4379](info): set up 2 sockets
Jun 19 08:24:28 dist-test-slave-5rhg krb5kdc[4379](info): commencing operation
krb5kdc: starting...
W20260619 08:24:29.230085 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:24:29 dist-test-slave-5rhg krb5kdc[4379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:24:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:24:29Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.25.127.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.25.127.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.25.127.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/rangeradmin_127.25.127.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.25.127.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/rangeradmin_127.25.127.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.25.127.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.25.127.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.25.127.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/rangerlookup_127.25.127.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.25.127.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/rangerlookup_127.25.127.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.127.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.25.127.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.127.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/HTTP_127.25.127.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.127.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/HTTP_127.25.127.148@KRBTEST.COM.keytab.
I20260619 08:24:29.396991 26110 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task2BFFaB/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/postgres -l logfile start
2026-06-19 08:24:31.765 UTC [4409] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 08:24:31.765 UTC [4409] LOG: listening on IPv4 address "127.25.127.148", port 34917
2026-06-19 08:24:31.768 UTC [4409] LOG: listening on Unix socket "/tmp/.s.PGSQL.34917"
2026-06-19 08:24:31.803 UTC [4412] LOG: database system was shut down at 2026-06-19 08:24:30 UTC
2026-06-19 08:24:31.807 UTC [4409] LOG: database system is ready to accept connections
I20260619 08:24:32.765929 26110 mini_postgres.cc:96] Postgres bound to 34917
2026-06-19 08:24:32.772 UTC [4420] FATAL: database "slave" does not exist
127.25.127.148:34917 - accepting connections
I20260619 08:24:32.772739 26110 mini_ranger.cc:161] Starting Ranger...
I20260619 08:24:32.788846 26110 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 08:24:33.245352 26110 mini_ranger.cc:87] Created ranger Postgres database
I20260619 08:24:33.245442 26110 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ranger-admin
2026-06-19 08:24:33,576 [I] DB FLAVOR :POSTGRES
2026-06-19 08:24:33,576 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 08:24:33,576 [I] Checking connection..
2026-06-19 08:24:33,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 08:24:33,829 [I] Checking connection passed.
2026-06-19 08:24:33,830 [I] --------- Verifying version history table ---------
2026-06-19 08:24:33,830 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 08:24:34,101 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 08:24:34,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 08:24:34,343 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 08:24:34,343 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 08:24:34,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 08:24:34.578 UTC [4484] WARNING: there is no transaction in progress
2026-06-19 08:24:34,585 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 08:24:34,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 08:24:34,843 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 08:24:34,843 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 08:24:34,843 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 08:24:35,087 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 08:24:35,336 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5rhg.c.gcp-upstream.internal','N') ;"
2026-06-19 08:24:35,572 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 08:24:35,834 [I] Table x_portal_user does not exist in database ranger
2026-06-19 08:24:35,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 08:24:36,074 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 08:24:36,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 08:24:36,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 08:24:36,588 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 08:24:36,588 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 08:24:37.552 UTC [4592] WARNING: there is no transaction in progress
2026-06-19 08:24:37.569 UTC [4592] WARNING: there is no transaction in progress
2026-06-19 08:24:37.586 UTC [4592] WARNING: there is no transaction in progress
2026-06-19 08:24:37.602 UTC [4592] WARNING: there is no transaction in progress
2026-06-19 08:24:37.619 UTC [4592] WARNING: there is no transaction in progress
2026-06-19 08:24:37.638 UTC [4592] WARNING: there is no transaction in progress
2026-06-19 08:24:37.694 UTC [4592] WARNING: there is no transaction in progress
2026-06-19 08:24:37.701 UTC [4592] WARNING: there is no transaction in progress
2026-06-19 08:24:37.716 UTC [4592] WARNING: there is no transaction in progress
2026-06-19 08:24:37.727 UTC [4592] WARNING: there is no transaction in progress
2026-06-19 08:24:37.737 UTC [4592] WARNING: there is no transaction in progress
2026-06-19 08:24:37.751 UTC [4592] WARNING: there is no transaction in progress
2026-06-19 08:24:37.761 UTC [4592] WARNING: there is no transaction in progress
2026-06-19 08:24:37.768 UTC [4592] WARNING: there is no transaction in progress
2026-06-19 08:24:37.774 UTC [4592] WARNING: there is no transaction in progress
2026-06-19 08:24:37.780 UTC [4592] WARNING: there is no transaction in progress
2026-06-19 08:24:38,442 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 08:24:38,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 08:24:38,682 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 08:24:38,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 08:24:38,926 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 08:24:38,927 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 08:24:39,184 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 08:24:39,184 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 08:24:39,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 08:24:39,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5rhg.c.gcp-upstream.internal';"
2026-06-19 08:24:39,938 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 08:24:39,938 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 08:24:40,202 [I] DB_PATCHES have already been applied
I20260619 08:24:40.208220 26110 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ranger-admin:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task2BFFaB/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 08:24:40.208303 26110 mini_ranger.cc:193] Using host: 127.25.127.148
I20260619 08:24:40.211439 26110 mini_ranger.cc:227] Ranger admin URL: http://127.25.127.148:51093
Jun 19, 2026 8:24:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:24:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:24:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 8:24:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 8:24:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 8:24:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 8:24:41 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 08:24:41.336277 26110 mini_ranger.cc:160] Time spent starting Ranger: real 8.564s user 0.007s sys 0.002s
Jun 19, 2026 8:25:02 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 8:25:02 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Jun 19, 2026 8:25:02 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 8:25:02 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 8:25:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 8:25:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 8:25:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 8:25:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 8:25:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 8:25:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 8:25:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 8:25:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 8:25:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 8:25:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 8:25:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 8:25:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 8:25:02 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 8:25:03 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 08:25:03.940405 26110 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.25.127.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.25.127.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.25.127.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/rangerkms_127.25.127.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.25.127.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/rangerkms_127.25.127.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.127.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.25.127.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.127.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/HTTP_127.25.127.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.127.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/HTTP_127.25.127.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 08:25:04 dist-test-slave-5rhg krb5kdc[4379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857504, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 08:25:04.019039 26110 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 08:25:04.028491 26110 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 08:25:04.839916 26110 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 08:25:04.840004 26110 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ranger-kms
I20260619 08:25:04.840026 26110 mini_ranger_kms.cc:227] Using postgres at 127.25.127.148:34917
2026-06-19 08:25:05,302 [I] DB FLAVOR :POSTGRES
2026-06-19 08:25:05,302 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 08:25:05,302 [I] Checking connection
2026-06-19 08:25:05,302 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 08:25:05,606 [I] connection success
2026-06-19 08:25:05,606 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 08:25:05,606 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 08:25:05,606 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 08:25:05,899 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 08:25:05,899 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 08:25:05,900 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 08:25:05,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:34917/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 08:25:06,211 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 08:25:06.732172 26110 mini_ranger_kms.cc:310] Created kms service
I20260619 08:25:06.914310 26110 mini_ranger_kms.cc:326] Created kudu user
I20260619 08:25:06.976667 26110 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 08:25:06 dist-test-slave-5rhg krb5kdc[4379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857504, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
I20260619 08:25:07.469370 26110 mini_ranger_kms.cc:384] Added ranger policy
I20260619 08:25:07.469497 26110 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ranger-kms:/tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 08:25:07.469527 26110 mini_ranger_kms.cc:240] Using host: 127.25.127.148
I20260619 08:25:07.472774 26110 mini_ranger_kms.cc:276] Ranger KMS PID: 4773
I20260619 08:25:07.472913 26110 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.25.127.148:42851
Jun 19, 2026 8:25:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:25:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:25:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 8:25:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 8:25:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 8:25:09 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 08:25:09.687247 26110 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.668s user 0.000s sys 0.286s
I20260619 08:25:09.687374 26110 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 08:25:09.687479 26110 mini_ranger_kms.cc:401] 127.25.127.148:42851/kms/v1/keys
Jun 19 08:25:09 dist-test-slave-5rhg krb5kdc[4379](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781857509, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.127.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 08:25:10 dist-test-slave-5rhg krb5kdc[4379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857509, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.127.148@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
Jun 19 08:25:10 dist-test-slave-5rhg krb5kdc[4379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857509, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.127.148@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
Jun 19 08:25:13 dist-test-slave-5rhg krb5kdc[4379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:25:13.227288 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:44325
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:32891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:44325
--ranger_config_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.127.148:42851/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:25:13.354331 4823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:25:13.354584 4823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:25:13.354609 4823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:25:13.355913 4823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:25:13.355964 4823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:25:13.355979 4823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:25:13.355993 4823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:25:13.356006 4823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:25:13.357904 4823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:32891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/master-0/wal
--ranger_kms_url=127.25.127.148:42851/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:44325
--ranger_config_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:44325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:25:13.358181 4823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:25:13.358449 4823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:25:13.364984 4828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:25:13.365154 4829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:25:13.365475 4823 server_base.cc:1034] running on GCE node
W20260619 08:25:13.365154 4831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:25:13.366183 4823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:25:13.366664 4823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:25:13.367853 4823 hybrid_clock.cc:648] HybridClock initialized: now 1781857513367833 us; error 67 us; skew 500 ppm
Jun 19 08:25:13 dist-test-slave-5rhg krb5kdc[4379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857513, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:25:13.375782 4823 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:25:13.376487 4823 webserver.cc:460] Webserver started at http://127.25.127.190:33299/ using document root <none> and password file <none>
I20260619 08:25:13.376786 4823 fs_manager.cc:359] Metadata directory not provided
I20260619 08:25:13.376858 4823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:25:13.377032 4823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:25:13 dist-test-slave-5rhg krb5kdc[4379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857513, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
I20260619 08:25:13.441224 4823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "b9a4e9a4f76545509307f74124596708"
format_stamp: "Formatted at 2026-06-19 08:25:13 on dist-test-slave-5rhg"
server_key: "8f0aa04bb522a508c2a4659222e9cd5a"
server_key_iv: "a485bbe64dbf3a4018ca5c7663c3da5e"
server_key_version: "kuduclusterkey@0"
I20260619 08:25:13.441689 4823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "b9a4e9a4f76545509307f74124596708"
format_stamp: "Formatted at 2026-06-19 08:25:13 on dist-test-slave-5rhg"
server_key: "8f0aa04bb522a508c2a4659222e9cd5a"
server_key_iv: "a485bbe64dbf3a4018ca5c7663c3da5e"
server_key_version: "kuduclusterkey@0"
I20260619 08:25:13.443221 4823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:25:13.458416 4838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:25:13.458695 4823 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:25:13.458850 4823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "b9a4e9a4f76545509307f74124596708"
format_stamp: "Formatted at 2026-06-19 08:25:13 on dist-test-slave-5rhg"
server_key: "8f0aa04bb522a508c2a4659222e9cd5a"
server_key_iv: "a485bbe64dbf3a4018ca5c7663c3da5e"
server_key_version: "kuduclusterkey@0"
I20260619 08:25:13.458966 4823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:25:13.467850 4823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:25:13.469794 4823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:25:13.469952 4823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:25:13.475040 4823 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:44325
I20260619 08:25:13.475096 4890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:44325 every 8 connection(s)
I20260619 08:25:13.475486 4823 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:25:13.476263 4891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:25:13.478682 4891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708: Bootstrap starting.
I20260619 08:25:13.479334 4891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708: Neither blocks nor log segments found. Creating new log.
I20260619 08:25:13.479650 4891 log.cc:824] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708: Log is configured to *not* fsync() on all Append() calls
I20260619 08:25:13.480451 4891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708: No bootstrap required, opened a new log
I20260619 08:25:13.482565 4891 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a4e9a4f76545509307f74124596708" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44325 } }
I20260619 08:25:13.482734 4891 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:25:13.482766 4891 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9a4e9a4f76545509307f74124596708, State: Initialized, Role: FOLLOWER
I20260619 08:25:13.482944 4891 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a4e9a4f76545509307f74124596708" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44325 } }
I20260619 08:25:13.483022 4891 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:25:13.483067 4891 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:25:13.483115 4891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:25:13.483742 4891 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a4e9a4f76545509307f74124596708" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44325 } }
I20260619 08:25:13.483844 4891 leader_election.cc:302] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9a4e9a4f76545509307f74124596708; no voters:
I20260619 08:25:13.484093 4891 leader_election.cc:288] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:25:13.484122 4894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:25:13.484344 4894 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708 [term 1 LEADER]: Becoming Leader. State: Replica: b9a4e9a4f76545509307f74124596708, State: Running, Role: LEADER
I20260619 08:25:13.484408 4891 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:25:13.484494 4894 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a4e9a4f76545509307f74124596708" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44325 } }
I20260619 08:25:13.484862 4896 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9a4e9a4f76545509307f74124596708" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a4e9a4f76545509307f74124596708" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44325 } } }
I20260619 08:25:13.484963 4896 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708 [sys.catalog]: This master's current role is: LEADER
I20260619 08:25:13.484849 4894 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9a4e9a4f76545509307f74124596708. Latest consensus state: current_term: 1 leader_uuid: "b9a4e9a4f76545509307f74124596708" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a4e9a4f76545509307f74124596708" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44325 } } }
I20260619 08:25:13.485194 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 4823
I20260619 08:25:13.485285 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:25:13.485502 26110 external_mini_cluster.cc:1428] Setting key a5208a619f088f22e88e4fb808c3e770
I20260619 08:25:13.485298 4894 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708 [sys.catalog]: This master's current role is: LEADER
I20260619 08:25:13.485666 4900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:25:13.486184 4900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:25:13.486723 4891 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 08:25:13.495276 4900 catalog_manager.cc:1347] Generated new cluster ID: 5c2e9a45c29c4529bef51270c1c83c65
I20260619 08:25:13.495427 4900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:25:13 dist-test-slave-5rhg krb5kdc[4379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:25:13.503453 4907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:13.486177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45168 (local address 127.25.127.190:44325)
0619 08:25:13.486561 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0619 08:25:13.486572 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:25:13.486605 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:25:13.486973 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:25:13.486987 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:25:13.487063 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:25:13.487223 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:25:13.488027 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:13.492725 (+ 4698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:13.493648 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:13.494576 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:13.498126 (+ 3550us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:13.498333 (+ 207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:25:13.498337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:25:13.498338 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:25:13.498348 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:25:13.501069 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:25:13.501546 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:25:13.501550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:25:13.501551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:25:13.501653 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:25:13.501852 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:25:13.501853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:25:13.501854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:25:13.502316 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:25:13.502912 (+ 596us) server_negotiation.cc:1035] Waiting for connection context
0619 08:25:13.502985 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 08:25:13.503204 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":83,"threads_started":1}
I20260619 08:25:13.506135 4900 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:25:13.507592 4900 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:25:13.514864 4900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708: Generated new TSK 0
I20260619 08:25:13.515247 4900 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
I20260619 08:25:14.270586 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.127.148:42851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:44325
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:32891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:25:14.385949 4936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:25:14.386236 4936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:25:14.386272 4936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:25:14.387995 4936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:25:14.388058 4936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:25:14.388098 4936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:25:14.390239 4936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:32891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.25.127.148:42851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:44325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:25:14.390482 4936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:25:14.390784 4936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:25:14.395977 4941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:25:14.396096 4942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:25:14.395970 4944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:25:14.396674 4936 server_base.cc:1034] running on GCE node
I20260619 08:25:14.396922 4936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:25:14.397259 4936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:25:14.398474 4936 hybrid_clock.cc:648] HybridClock initialized: now 1781857514398444 us; error 39 us; skew 500 ppm
Jun 19 08:25:14 dist-test-slave-5rhg krb5kdc[4379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857514, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:25:14.402078 4936 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:25:14.402631 4936 webserver.cc:460] Webserver started at http://127.25.127.129:41231/ using document root <none> and password file <none>
I20260619 08:25:14.402884 4936 fs_manager.cc:359] Metadata directory not provided
I20260619 08:25:14.402948 4936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:25:14.403126 4936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:25:14 dist-test-slave-5rhg krb5kdc[4379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857514, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
I20260619 08:25:14.418143 4936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "a73ab8d027bc4a268948b419aead9f02"
format_stamp: "Formatted at 2026-06-19 08:25:14 on dist-test-slave-5rhg"
server_key: "a2b5792e97a1ca552b7b4efcef90b05a"
server_key_iv: "7f0b8474663ad4ad713e39b92870e53f"
server_key_version: "kuduclusterkey@0"
I20260619 08:25:14.418596 4936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "a73ab8d027bc4a268948b419aead9f02"
format_stamp: "Formatted at 2026-06-19 08:25:14 on dist-test-slave-5rhg"
server_key: "a2b5792e97a1ca552b7b4efcef90b05a"
server_key_iv: "7f0b8474663ad4ad713e39b92870e53f"
server_key_version: "kuduclusterkey@0"
I20260619 08:25:14.420850 4936 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260619 08:25:14.436322 4951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:25:14.440403 4936 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260619 08:25:14.440627 4936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "a73ab8d027bc4a268948b419aead9f02"
format_stamp: "Formatted at 2026-06-19 08:25:14 on dist-test-slave-5rhg"
server_key: "a2b5792e97a1ca552b7b4efcef90b05a"
server_key_iv: "7f0b8474663ad4ad713e39b92870e53f"
server_key_version: "kuduclusterkey@0"
I20260619 08:25:14.440831 4936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:25:14.458246 4936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:25:14.461030 4936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:25:14.461197 4936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:25:14.461448 4936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:25:14.461835 4936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:25:14.461885 4936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:25:14.461917 4936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:25:14.461935 4936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:25:14.471029 4936 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:45405
I20260619 08:25:14.471601 4936 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:25:14.472379 5066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:45405 every 8 connection(s)
I20260619 08:25:14.480830 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 4936
I20260619 08:25:14.480995 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
Jun 19 08:25:14 dist-test-slave-5rhg krb5kdc[4379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857514, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:25:14.481204 26110 external_mini_cluster.cc:1428] Setting key 889f5304bd8be07f015164d6c5ba9a70
I20260619 08:25:14.485709 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:14.472512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:37335 (local address 127.25.127.190:44325)
0619 08:25:14.472686 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 08:25:14.472688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:25:14.472976 (+ 288us) server_negotiation.cc:407] Connection header received
0619 08:25:14.473954 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:25:14.473955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:25:14.473971 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:25:14.474046 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:25:14.475316 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:14.476296 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:14.477986 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:14.478301 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:14.478344 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:14.481741 (+ 3397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:25:14.481742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:25:14.481742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:25:14.481751 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:25:14.483937 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:25:14.484580 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:25:14.484581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:25:14.484581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:25:14.484648 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:25:14.484928 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:25:14.484929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:25:14.484930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:25:14.485069 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:25:14.485110 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 08:25:14.485509 (+ 399us) server_negotiation.cc:299] Negotiation successful
0619 08:25:14.485603 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260619 08:25:14.486223 5070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:14.472512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:44325 (local address 127.25.127.129:37335)
0619 08:25:14.472823 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0619 08:25:14.472871 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 08:25:14.473817 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:25:14.474282 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:25:14.474287 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:25:14.474717 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:25:14.475175 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:25:14.475183 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:25:14.476448 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:25:14.476449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:25:14.477880 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:25:14.477883 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:25:14.478449 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:25:14.478450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:25:14.478651 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:14.479422 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:25:14.479433 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:25:14.481616 (+ 2183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:25:14.484063 (+ 2447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:25:14.484079 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:25:14.484082 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:25:14.484481 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:25:14.484719 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:25:14.484720 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:25:14.484720 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:25:14.484860 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:25:14.485155 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:25:14.485161 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:25:14.485340 (+ 179us) client_negotiation.cc:769] Sending connection context
0619 08:25:14.485815 (+ 475us) client_negotiation.cc:240] Negotiation successful
0619 08:25:14.485985 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":208,"threads_started":1}
I20260619 08:25:14.486779 5067 heartbeater.cc:347] Connected to a master server at 127.25.127.190:44325
I20260619 08:25:14.486894 5067 heartbeater.cc:464] Registering TS with master...
I20260619 08:25:14.487092 5067 heartbeater.cc:511] Master 127.25.127.190:44325 requested a full tablet report, sending...
I20260619 08:25:14.487746 4855 ts_manager.cc:194] Registered new tserver with Master: a73ab8d027bc4a268948b419aead9f02 (127.25.127.129:45405)
I20260619 08:25:14.488727 4855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:37335
WARNING: no policy specified for kudu/127.25.127.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
I20260619 08:25:14.517084 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.127.148:42851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:44325
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:32891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:25:14.634037 5073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:25:14.634269 5073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:25:14.634310 5073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:25:14.635522 5073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:25:14.635680 5073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:25:14.635753 5073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:25:14.637516 5073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:32891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.25.127.148:42851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:44325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:25:14.637791 5073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:25:14.638055 5073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:25:14.643069 5080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:25:14.643025 5084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:25:14.643431 5081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:25:14.643973 5073 server_base.cc:1034] running on GCE node
I20260619 08:25:14.644243 5073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:25:14.644543 5073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:25:14.645713 5073 hybrid_clock.cc:648] HybridClock initialized: now 1781857514645693 us; error 38 us; skew 500 ppm
Jun 19 08:25:14 dist-test-slave-5rhg krb5kdc[4379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857514, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:25:14.648806 5073 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:25:14.649302 5073 webserver.cc:460] Webserver started at http://127.25.127.130:33261/ using document root <none> and password file <none>
I20260619 08:25:14.649606 5073 fs_manager.cc:359] Metadata directory not provided
I20260619 08:25:14.649714 5073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:25:14.649874 5073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:25:14 dist-test-slave-5rhg krb5kdc[4379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857514, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
I20260619 08:25:14.664507 5073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "1f73d6d74643494c92b706b2d89e0e27"
format_stamp: "Formatted at 2026-06-19 08:25:14 on dist-test-slave-5rhg"
server_key: "462730b34f63d5a552f593005c3a71f8"
server_key_iv: "875724f0926c078cbfefc16cc04ea6a5"
server_key_version: "kuduclusterkey@0"
I20260619 08:25:14.665107 5073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "1f73d6d74643494c92b706b2d89e0e27"
format_stamp: "Formatted at 2026-06-19 08:25:14 on dist-test-slave-5rhg"
server_key: "462730b34f63d5a552f593005c3a71f8"
server_key_iv: "875724f0926c078cbfefc16cc04ea6a5"
server_key_version: "kuduclusterkey@0"
I20260619 08:25:14.673025 5073 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.000s
Jun 19 08:25:14 dist-test-slave-5rhg krb5kdc[4379](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781857514, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:25:14.700282 5091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:25:14.700651 5073 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.001s sys 0.000s
I20260619 08:25:14.700834 5073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "1f73d6d74643494c92b706b2d89e0e27"
format_stamp: "Formatted at 2026-06-19 08:25:14 on dist-test-slave-5rhg"
server_key: "462730b34f63d5a552f593005c3a71f8"
server_key_iv: "875724f0926c078cbfefc16cc04ea6a5"
server_key_version: "kuduclusterkey@0"
I20260619 08:25:14.701031 5073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:25:14.711829 5073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:25:14.714443 5073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:25:14.714670 5073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:25:14.714989 5073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:25:14.715423 5073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:25:14.715519 5073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:25:14.715598 5073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:25:14.715659 5073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:25:14.722210 5073 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:45173
I20260619 08:25:14.722718 5073 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:25:14.726714 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 5073
I20260619 08:25:14.726822 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:25:14.727016 26110 external_mini_cluster.cc:1428] Setting key 6c0d1a996549ff8f78dfb92a76105bd2
I20260619 08:25:14.737478 5206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:45173 every 8 connection(s)
WARNING: no policy specified for kudu/127.25.127.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.131@KRBTEST.COM" created.
Jun 19 08:25:14 dist-test-slave-5rhg krb5kdc[4379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857514, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:25:14.751669 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:14.738681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:50847 (local address 127.25.127.190:44325)
0619 08:25:14.738745 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 08:25:14.738748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:25:14.740455 (+ 1707us) server_negotiation.cc:407] Connection header received
0619 08:25:14.741180 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:25:14.741181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:25:14.741194 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:25:14.741225 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:25:14.742136 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:14.743014 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:14.744849 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:14.745166 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:14.745253 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:14.748170 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:25:14.748172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:25:14.748172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:25:14.748180 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:25:14.749986 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:25:14.750524 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:25:14.750525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:25:14.750525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:25:14.750588 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:25:14.750881 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:25:14.750881 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:25:14.750882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:25:14.751021 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:25:14.751065 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 08:25:14.751473 (+ 408us) server_negotiation.cc:299] Negotiation successful
0619 08:25:14.751566 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 08:25:14.752532 5210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:14.738289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:44325 (local address 127.25.127.130:50847)
0619 08:25:14.740282 (+ 1993us) negotiation.cc:107] Waiting for socket to connect
0619 08:25:14.740350 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0619 08:25:14.741040 (+ 690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:25:14.741459 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:25:14.741465 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:25:14.741719 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:25:14.742016 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:25:14.742024 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:25:14.743386 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:25:14.743387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:25:14.744741 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:25:14.744744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:25:14.745363 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:25:14.745364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:25:14.745523 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:14.746184 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:25:14.746195 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:25:14.748032 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:25:14.750085 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:25:14.750092 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:25:14.750094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:25:14.750427 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:25:14.750663 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:25:14.750664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:25:14.750664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:25:14.750801 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:25:14.751112 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:25:14.751119 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:25:14.751291 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 08:25:14.752089 (+ 798us) client_negotiation.cc:240] Negotiation successful
0619 08:25:14.752283 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1940,"thread_start_us":172,"threads_started":1}
I20260619 08:25:14.753262 5207 heartbeater.cc:347] Connected to a master server at 127.25.127.190:44325
I20260619 08:25:14.753369 5207 heartbeater.cc:464] Registering TS with master...
I20260619 08:25:14.753559 5207 heartbeater.cc:511] Master 127.25.127.190:44325 requested a full tablet report, sending...
I20260619 08:25:14.753930 4855 ts_manager.cc:194] Registered new tserver with Master: 1f73d6d74643494c92b706b2d89e0e27 (127.25.127.130:45173)
I20260619 08:25:14.754319 4855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:50847
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
I20260619 08:25:14.769377 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.127.148:42851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:44325
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:32891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:25:14.899387 5212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:25:14.899766 5212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:25:14.899856 5212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:25:14.901775 5212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:25:14.901918 5212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:25:14.902002 5212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:25:14.904628 5212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:32891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.25.127.148:42851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:44325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:25:14.905215 5212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:25:14.905556 5212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:25:14.914767 5219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:25:14.915145 5212 server_base.cc:1034] running on GCE node
W20260619 08:25:14.914788 5218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:25:14.915537 5221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:25:14.915827 5212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:25:14.916102 5212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:25:14.917340 5212 hybrid_clock.cc:648] HybridClock initialized: now 1781857514917312 us; error 33 us; skew 500 ppm
Jun 19 08:25:14 dist-test-slave-5rhg krb5kdc[4379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857514, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:25:14.920697 5212 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:25:14.921231 5212 webserver.cc:460] Webserver started at http://127.25.127.131:35509/ using document root <none> and password file <none>
I20260619 08:25:14.921485 5212 fs_manager.cc:359] Metadata directory not provided
I20260619 08:25:14.921532 5212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:25:14.921654 5212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:25:14 dist-test-slave-5rhg krb5kdc[4379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857514, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
I20260619 08:25:14.942785 5212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "dcc67942d65044e789ca5675e4bbf61c"
format_stamp: "Formatted at 2026-06-19 08:25:14 on dist-test-slave-5rhg"
server_key: "bed749f1606bce56d72b69044075e84b"
server_key_iv: "95c8d3202fd534a19ca32533b3bc4403"
server_key_version: "kuduclusterkey@0"
I20260619 08:25:14.943428 5212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "dcc67942d65044e789ca5675e4bbf61c"
format_stamp: "Formatted at 2026-06-19 08:25:14 on dist-test-slave-5rhg"
server_key: "bed749f1606bce56d72b69044075e84b"
server_key_iv: "95c8d3202fd534a19ca32533b3bc4403"
server_key_version: "kuduclusterkey@0"
I20260619 08:25:14.946492 5212 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260619 08:25:14.976452 5228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:25:14.976750 5212 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20260619 08:25:14.976846 5212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "dcc67942d65044e789ca5675e4bbf61c"
format_stamp: "Formatted at 2026-06-19 08:25:14 on dist-test-slave-5rhg"
server_key: "bed749f1606bce56d72b69044075e84b"
server_key_iv: "95c8d3202fd534a19ca32533b3bc4403"
server_key_version: "kuduclusterkey@0"
I20260619 08:25:14.976945 5212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:25:15.001726 5212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:25:15.004603 5212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:25:15.004773 5212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:25:15.005029 5212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:25:15.005424 5212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:25:15.005467 5212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:25:15.005498 5212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:25:15.005515 5212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:25:15.017757 5212 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:46137
I20260619 08:25:15.017874 5342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:46137 every 8 connection(s)
I20260619 08:25:15.018440 5212 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:25:15.019397 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 5212
I20260619 08:25:15.019495 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:25:15.019678 26110 external_mini_cluster.cc:1428] Setting key 94fd63db4a41e47cfd01432e6a5fc261
Jun 19 08:25:15 dist-test-slave-5rhg krb5kdc[4379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857514, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:25:15.031514 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:15.019663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:36793 (local address 127.25.127.190:44325)
0619 08:25:15.019716 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:25:15.019719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:25:15.020408 (+ 689us) server_negotiation.cc:407] Connection header received
0619 08:25:15.021142 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:25:15.021143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:25:15.021155 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:25:15.021214 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:25:15.022253 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.022920 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.024580 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.024859 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.024902 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:15.027863 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:25:15.027864 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:25:15.027865 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:25:15.027872 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:25:15.029633 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:25:15.030149 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:25:15.030150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:25:15.030150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:25:15.030219 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:25:15.030504 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:25:15.030505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:25:15.030505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:25:15.030638 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:25:15.030677 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 08:25:15.031317 (+ 640us) server_negotiation.cc:299] Negotiation successful
0619 08:25:15.031421 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:25:15.032446 5345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:15.019354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:44325 (local address 127.25.127.131:36793)
0619 08:25:15.020269 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0619 08:25:15.020323 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 08:25:15.021021 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:25:15.021364 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:25:15.021369 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:25:15.021769 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:25:15.022146 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:25:15.022153 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.023037 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.023038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:25:15.024480 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:25:15.024483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.024992 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.024993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:25:15.025179 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:15.025939 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:25:15.025949 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:25:15.027751 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:25:15.029729 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:25:15.029736 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:25:15.029739 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:25:15.030057 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:25:15.030286 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:25:15.030286 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:25:15.030287 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:25:15.030436 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:25:15.030721 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:25:15.030726 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:25:15.031162 (+ 436us) client_negotiation.cc:769] Sending connection context
0619 08:25:15.031979 (+ 817us) client_negotiation.cc:240] Negotiation successful
0619 08:25:15.032189 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"thread_start_us":95,"threads_started":1}
I20260619 08:25:15.033540 5343 heartbeater.cc:347] Connected to a master server at 127.25.127.190:44325
I20260619 08:25:15.033697 5343 heartbeater.cc:464] Registering TS with master...
I20260619 08:25:15.033964 5343 heartbeater.cc:511] Master 127.25.127.190:44325 requested a full tablet report, sending...
I20260619 08:25:15.034636 4855 ts_manager.cc:194] Registered new tserver with Master: dcc67942d65044e789ca5675e4bbf61c (127.25.127.131:46137)
I20260619 08:25:15.035601 4855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:36793
I20260619 08:25:15.043462 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:25:15.063424 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:15.045535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45176 (local address 127.25.127.190:44325)
0619 08:25:15.045587 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:25:15.045589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:25:15.054641 (+ 9052us) server_negotiation.cc:407] Connection header received
0619 08:25:15.054797 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:25:15.054798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:25:15.054812 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:25:15.054848 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:25:15.055815 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.056589 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.057722 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.058001 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.058054 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:15.059900 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:25:15.059901 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:25:15.059902 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:25:15.059911 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:25:15.061886 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:25:15.062365 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:25:15.062366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:25:15.062366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:25:15.062435 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:25:15.062671 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:25:15.062672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:25:15.062672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:25:15.062832 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:25:15.062914 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 08:25:15.063225 (+ 311us) server_negotiation.cc:299] Negotiation successful
0619 08:25:15.063317 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:25:15.065798 4855 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45176:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:25:15.068403 4855 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:25:15.077240 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:15.071640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:46137 (local address 127.0.0.1:55064)
0619 08:25:15.072760 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0619 08:25:15.072770 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:25:15.072828 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:25:15.073097 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:25:15.073100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:25:15.073103 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:25:15.073179 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:25:15.073181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.074473 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.074474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:25:15.075836 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:25:15.075838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.076923 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.076924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:25:15.077016 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:15.077030 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 08:25:15.077094 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 08:25:15.077148 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1102,"thread_start_us":104,"threads_started":1}
I20260619 08:25:15.077809 5359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:15.071915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55064 (local address 127.25.127.131:46137)
0619 08:25:15.072139 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 08:25:15.072145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:25:15.072852 (+ 707us) server_negotiation.cc:407] Connection header received
0619 08:25:15.072906 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:25:15.072909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:25:15.072926 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:25:15.073008 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:25:15.073257 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.074371 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.075959 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.076729 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.076795 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:15.076858 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:25:15.077682 (+ 824us) server_negotiation.cc:299] Negotiation successful
0619 08:25:15.077728 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":79,"threads_started":1}
I20260619 08:25:15.077915 5358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:15.071866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:45173 (local address 127.0.0.1:33010)
0619 08:25:15.072910 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0619 08:25:15.072919 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:25:15.072954 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:25:15.073328 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:25:15.073329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:25:15.073331 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:25:15.073384 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:25:15.073386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.075161 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.075162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:25:15.076264 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:25:15.076267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.077663 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.077664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:25:15.077743 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:15.077748 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:25:15.077786 (+ 38us) client_negotiation.cc:240] Negotiation successful
0619 08:25:15.077829 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1030,"thread_start_us":88,"threads_started":1}
I20260619 08:25:15.078876 5360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:15.072045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33010 (local address 127.25.127.130:45173)
0619 08:25:15.073078 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0619 08:25:15.073083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:25:15.073094 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:25:15.073137 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:25:15.073139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:25:15.073156 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:25:15.073238 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:25:15.073474 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.075045 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.076404 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.077545 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.078613 (+ 1068us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:15.078705 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:25:15.078737 (+ 32us) server_negotiation.cc:299] Negotiation successful
0619 08:25:15.078783 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":994,"thread_start_us":131,"threads_started":1}
I20260619 08:25:15.078890 5276 tablet_service.cc:1467] Processing CreateTablet for tablet 092dd5796e234ebca7327e354f74b926 (DEFAULT_TABLE table=test-table [id=c274eff39bb743ddbbe3ecc35405f787]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:25:15.079299 5276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 092dd5796e234ebca7327e354f74b926. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:25:15.079617 5140 tablet_service.cc:1467] Processing CreateTablet for tablet 092dd5796e234ebca7327e354f74b926 (DEFAULT_TABLE table=test-table [id=c274eff39bb743ddbbe3ecc35405f787]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:25:15.080005 5140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 092dd5796e234ebca7327e354f74b926. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:25:15.084276 5357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:15.071628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:45405 (local address 127.0.0.1:46574)
0619 08:25:15.072447 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0619 08:25:15.072493 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 08:25:15.072603 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:25:15.079787 (+ 7184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:25:15.079789 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:25:15.079792 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:25:15.079840 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:25:15.079842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.081228 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.081229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:25:15.082296 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:25:15.082300 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.083971 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.083972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:25:15.084043 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:15.084049 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 08:25:15.084085 (+ 36us) client_negotiation.cc:240] Negotiation successful
0619 08:25:15.084130 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":84,"threads_started":1}
I20260619 08:25:15.084805 5361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:15.072092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46574 (local address 127.25.127.129:45405)
0619 08:25:15.079238 (+ 7146us) server_negotiation.cc:206] Beginning negotiation
0619 08:25:15.079244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:25:15.079256 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:25:15.079297 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:25:15.079299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:25:15.079319 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:25:15.079928 (+ 609us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:25:15.080007 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.081102 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.082500 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.083653 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.084486 (+ 833us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:15.084572 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 08:25:15.084656 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 08:25:15.084715 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7099,"thread_start_us":90,"threads_started":1}
I20260619 08:25:15.085227 5363 tablet_bootstrap.cc:492] T 092dd5796e234ebca7327e354f74b926 P 1f73d6d74643494c92b706b2d89e0e27: Bootstrap starting.
I20260619 08:25:15.085489 5001 tablet_service.cc:1467] Processing CreateTablet for tablet 092dd5796e234ebca7327e354f74b926 (DEFAULT_TABLE table=test-table [id=c274eff39bb743ddbbe3ecc35405f787]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:25:15.085831 5001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 092dd5796e234ebca7327e354f74b926. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:25:15.086761 5363 tablet_bootstrap.cc:654] T 092dd5796e234ebca7327e354f74b926 P 1f73d6d74643494c92b706b2d89e0e27: Neither blocks nor log segments found. Creating new log.
I20260619 08:25:15.087598 5363 log.cc:824] T 092dd5796e234ebca7327e354f74b926 P 1f73d6d74643494c92b706b2d89e0e27: Log is configured to *not* fsync() on all Append() calls
I20260619 08:25:15.088052 5362 tablet_bootstrap.cc:492] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c: Bootstrap starting.
I20260619 08:25:15.088953 5365 tablet_bootstrap.cc:492] T 092dd5796e234ebca7327e354f74b926 P a73ab8d027bc4a268948b419aead9f02: Bootstrap starting.
I20260619 08:25:15.089164 5362 tablet_bootstrap.cc:654] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c: Neither blocks nor log segments found. Creating new log.
I20260619 08:25:15.089519 5362 log.cc:824] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c: Log is configured to *not* fsync() on all Append() calls
I20260619 08:25:15.089756 5365 tablet_bootstrap.cc:654] T 092dd5796e234ebca7327e354f74b926 P a73ab8d027bc4a268948b419aead9f02: Neither blocks nor log segments found. Creating new log.
I20260619 08:25:15.090119 5365 log.cc:824] T 092dd5796e234ebca7327e354f74b926 P a73ab8d027bc4a268948b419aead9f02: Log is configured to *not* fsync() on all Append() calls
I20260619 08:25:15.090134 5363 tablet_bootstrap.cc:492] T 092dd5796e234ebca7327e354f74b926 P 1f73d6d74643494c92b706b2d89e0e27: No bootstrap required, opened a new log
I20260619 08:25:15.090205 5363 ts_tablet_manager.cc:1397] T 092dd5796e234ebca7327e354f74b926 P 1f73d6d74643494c92b706b2d89e0e27: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 08:25:15.092438 5363 raft_consensus.cc:348] T 092dd5796e234ebca7327e354f74b926 P 1f73d6d74643494c92b706b2d89e0e27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73ab8d027bc4a268948b419aead9f02" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45405 } } peers { permanent_uuid: "1f73d6d74643494c92b706b2d89e0e27" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45173 } } peers { permanent_uuid: "dcc67942d65044e789ca5675e4bbf61c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46137 } }
I20260619 08:25:15.092923 5363 raft_consensus.cc:374] T 092dd5796e234ebca7327e354f74b926 P 1f73d6d74643494c92b706b2d89e0e27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:25:15.092979 5363 raft_consensus.cc:729] T 092dd5796e234ebca7327e354f74b926 P 1f73d6d74643494c92b706b2d89e0e27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f73d6d74643494c92b706b2d89e0e27, State: Initialized, Role: FOLLOWER
I20260619 08:25:15.093129 5363 consensus_queue.cc:260] T 092dd5796e234ebca7327e354f74b926 P 1f73d6d74643494c92b706b2d89e0e27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73ab8d027bc4a268948b419aead9f02" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45405 } } peers { permanent_uuid: "1f73d6d74643494c92b706b2d89e0e27" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45173 } } peers { permanent_uuid: "dcc67942d65044e789ca5675e4bbf61c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46137 } }
I20260619 08:25:15.093423 5363 ts_tablet_manager.cc:1428] T 092dd5796e234ebca7327e354f74b926 P 1f73d6d74643494c92b706b2d89e0e27: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:25:15.093555 5207 heartbeater.cc:503] Master 127.25.127.190:44325 was elected leader, sending a full tablet report...
I20260619 08:25:15.095702 5362 tablet_bootstrap.cc:492] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c: No bootstrap required, opened a new log
I20260619 08:25:15.095794 5362 ts_tablet_manager.cc:1397] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260619 08:25:15.099462 5362 raft_consensus.cc:348] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73ab8d027bc4a268948b419aead9f02" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45405 } } peers { permanent_uuid: "1f73d6d74643494c92b706b2d89e0e27" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45173 } } peers { permanent_uuid: "dcc67942d65044e789ca5675e4bbf61c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46137 } }
I20260619 08:25:15.099674 5362 raft_consensus.cc:374] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:25:15.099735 5362 raft_consensus.cc:729] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcc67942d65044e789ca5675e4bbf61c, State: Initialized, Role: FOLLOWER
I20260619 08:25:15.099897 5362 consensus_queue.cc:260] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73ab8d027bc4a268948b419aead9f02" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45405 } } peers { permanent_uuid: "1f73d6d74643494c92b706b2d89e0e27" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45173 } } peers { permanent_uuid: "dcc67942d65044e789ca5675e4bbf61c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46137 } }
I20260619 08:25:15.100245 5362 ts_tablet_manager.cc:1428] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260619 08:25:15.100272 5343 heartbeater.cc:503] Master 127.25.127.190:44325 was elected leader, sending a full tablet report...
I20260619 08:25:15.101227 5365 tablet_bootstrap.cc:492] T 092dd5796e234ebca7327e354f74b926 P a73ab8d027bc4a268948b419aead9f02: No bootstrap required, opened a new log
I20260619 08:25:15.101353 5365 ts_tablet_manager.cc:1397] T 092dd5796e234ebca7327e354f74b926 P a73ab8d027bc4a268948b419aead9f02: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20260619 08:25:15.104918 5365 raft_consensus.cc:348] T 092dd5796e234ebca7327e354f74b926 P a73ab8d027bc4a268948b419aead9f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73ab8d027bc4a268948b419aead9f02" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45405 } } peers { permanent_uuid: "1f73d6d74643494c92b706b2d89e0e27" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45173 } } peers { permanent_uuid: "dcc67942d65044e789ca5675e4bbf61c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46137 } }
I20260619 08:25:15.105222 5365 raft_consensus.cc:374] T 092dd5796e234ebca7327e354f74b926 P a73ab8d027bc4a268948b419aead9f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:25:15.105279 5365 raft_consensus.cc:729] T 092dd5796e234ebca7327e354f74b926 P a73ab8d027bc4a268948b419aead9f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a73ab8d027bc4a268948b419aead9f02, State: Initialized, Role: FOLLOWER
I20260619 08:25:15.105535 5365 consensus_queue.cc:260] T 092dd5796e234ebca7327e354f74b926 P a73ab8d027bc4a268948b419aead9f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73ab8d027bc4a268948b419aead9f02" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45405 } } peers { permanent_uuid: "1f73d6d74643494c92b706b2d89e0e27" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45173 } } peers { permanent_uuid: "dcc67942d65044e789ca5675e4bbf61c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46137 } }
I20260619 08:25:15.105983 5365 ts_tablet_manager.cc:1428] T 092dd5796e234ebca7327e354f74b926 P a73ab8d027bc4a268948b419aead9f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260619 08:25:15.106168 5067 heartbeater.cc:503] Master 127.25.127.190:44325 was elected leader, sending a full tablet report...
W20260619 08:25:15.222824 5068 tablet.cc:2369] T 092dd5796e234ebca7327e354f74b926 P a73ab8d027bc4a268948b419aead9f02: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:25:15.239297 5208 tablet.cc:2369] T 092dd5796e234ebca7327e354f74b926 P 1f73d6d74643494c92b706b2d89e0e27: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:25:15.247227 5369 raft_consensus.cc:482] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:25:15.247419 5369 raft_consensus.cc:504] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73ab8d027bc4a268948b419aead9f02" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45405 } } peers { permanent_uuid: "1f73d6d74643494c92b706b2d89e0e27" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45173 } } peers { permanent_uuid: "dcc67942d65044e789ca5675e4bbf61c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46137 } }
I20260619 08:25:15.247865 5369 leader_election.cc:288] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a73ab8d027bc4a268948b419aead9f02 (127.25.127.129:45405), 1f73d6d74643494c92b706b2d89e0e27 (127.25.127.130:45173)
Jun 19 08:25:15 dist-test-slave-5rhg krb5kdc[4379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857514, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
I20260619 08:25:15.252614 5345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:15.248074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:45405 (local address 127.25.127.131:58421)
0619 08:25:15.248122 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 08:25:15.248139 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 08:25:15.248245 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:25:15.248702 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:25:15.248703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:25:15.248706 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:25:15.248780 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:25:15.248782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.249698 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.249700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:25:15.250933 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:25:15.250936 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.252351 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.252352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:25:15.252438 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:15.252442 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:25:15.252492 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 08:25:15.252537 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 08:25:15.252892 5361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:15.248523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:58421 (local address 127.25.127.129:45405)
0619 08:25:15.248564 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:25:15.248566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:25:15.248575 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:25:15.248603 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:25:15.248604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:25:15.248615 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:25:15.248672 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:25:15.248874 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.249600 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.251077 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.251825 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.251893 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:15.251902 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:25:15.252776 (+ 874us) server_negotiation.cc:299] Negotiation successful
0619 08:25:15.252825 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:25:15.253180 5021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "092dd5796e234ebca7327e354f74b926" candidate_uuid: "dcc67942d65044e789ca5675e4bbf61c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a73ab8d027bc4a268948b419aead9f02" is_pre_election: true
I20260619 08:25:15.253345 5021 raft_consensus.cc:2454] T 092dd5796e234ebca7327e354f74b926 P a73ab8d027bc4a268948b419aead9f02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcc67942d65044e789ca5675e4bbf61c in term 0.
I20260619 08:25:15.253707 5230 leader_election.cc:302] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a73ab8d027bc4a268948b419aead9f02, dcc67942d65044e789ca5675e4bbf61c; no voters:
I20260619 08:25:15.254071 5369 raft_consensus.cc:2781] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:25:15.254132 5369 raft_consensus.cc:482] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:25:15.254154 5369 raft_consensus.cc:3037] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:25:15.254545 5360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:15.248303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:36737 (local address 127.25.127.130:45173)
0619 08:25:15.248344 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:25:15.248346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:25:15.249036 (+ 690us) server_negotiation.cc:407] Connection header received
0619 08:25:15.249127 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:25:15.249128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:25:15.249139 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:25:15.249166 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:25:15.249731 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.250471 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.253375 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.254010 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.254064 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:15.254073 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:25:15.254434 (+ 361us) server_negotiation.cc:299] Negotiation successful
0619 08:25:15.254475 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:25:15.254889 5371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:15.248770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:45173 (local address 127.25.127.131:36737)
0619 08:25:15.248974 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 08:25:15.248984 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:25:15.249060 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:25:15.249224 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:25:15.249225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:25:15.249226 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:25:15.249427 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:25:15.249429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.251555 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.251556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:25:15.253264 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:25:15.253267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.254227 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.254228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:25:15.254329 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:15.254333 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:25:15.254654 (+ 321us) client_negotiation.cc:240] Negotiation successful
0619 08:25:15.254701 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":109,"threads_started":1}
I20260619 08:25:15.255102 5369 raft_consensus.cc:504] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73ab8d027bc4a268948b419aead9f02" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45405 } } peers { permanent_uuid: "1f73d6d74643494c92b706b2d89e0e27" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45173 } } peers { permanent_uuid: "dcc67942d65044e789ca5675e4bbf61c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46137 } }
I20260619 08:25:15.255218 5369 leader_election.cc:288] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [CANDIDATE]: Term 1 election: Requested vote from peers a73ab8d027bc4a268948b419aead9f02 (127.25.127.129:45405), 1f73d6d74643494c92b706b2d89e0e27 (127.25.127.130:45173)
I20260619 08:25:15.255789 5160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "092dd5796e234ebca7327e354f74b926" candidate_uuid: "dcc67942d65044e789ca5675e4bbf61c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f73d6d74643494c92b706b2d89e0e27" is_pre_election: true
I20260619 08:25:15.255785 5159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "092dd5796e234ebca7327e354f74b926" candidate_uuid: "dcc67942d65044e789ca5675e4bbf61c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f73d6d74643494c92b706b2d89e0e27"
I20260619 08:25:15.255966 5160 raft_consensus.cc:2454] T 092dd5796e234ebca7327e354f74b926 P 1f73d6d74643494c92b706b2d89e0e27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcc67942d65044e789ca5675e4bbf61c in term 0.
I20260619 08:25:15.256381 5021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "092dd5796e234ebca7327e354f74b926" candidate_uuid: "dcc67942d65044e789ca5675e4bbf61c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a73ab8d027bc4a268948b419aead9f02"
I20260619 08:25:15.256481 5021 raft_consensus.cc:3037] T 092dd5796e234ebca7327e354f74b926 P a73ab8d027bc4a268948b419aead9f02 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:25:15.257448 5021 raft_consensus.cc:2454] T 092dd5796e234ebca7327e354f74b926 P a73ab8d027bc4a268948b419aead9f02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcc67942d65044e789ca5675e4bbf61c in term 1.
I20260619 08:25:15.257650 5230 leader_election.cc:302] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a73ab8d027bc4a268948b419aead9f02, dcc67942d65044e789ca5675e4bbf61c; no voters: 1f73d6d74643494c92b706b2d89e0e27
I20260619 08:25:15.257772 5369 raft_consensus.cc:2781] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:25:15.257925 5369 raft_consensus.cc:686] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [term 1 LEADER]: Becoming Leader. State: Replica: dcc67942d65044e789ca5675e4bbf61c, State: Running, Role: LEADER
I20260619 08:25:15.258037 5369 consensus_queue.cc:237] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73ab8d027bc4a268948b419aead9f02" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45405 } } peers { permanent_uuid: "1f73d6d74643494c92b706b2d89e0e27" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45173 } } peers { permanent_uuid: "dcc67942d65044e789ca5675e4bbf61c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46137 } }
I20260619 08:25:15.258913 4855 catalog_manager.cc:5495] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c reported cstate change: term changed from 0 to 1, leader changed from <none> to dcc67942d65044e789ca5675e4bbf61c (127.25.127.131). New cstate: current_term: 1 leader_uuid: "dcc67942d65044e789ca5675e4bbf61c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73ab8d027bc4a268948b419aead9f02" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f73d6d74643494c92b706b2d89e0e27" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcc67942d65044e789ca5675e4bbf61c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46137 } health_report { overall_health: HEALTHY } } }
W20260619 08:25:15.271726 5344 tablet.cc:2369] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:25:15.303874 5359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:15.300197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55080 (local address 127.25.127.131:46137)
0619 08:25:15.300253 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:25:15.300255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:25:15.300322 (+ 67us) server_negotiation.cc:407] Connection header received
0619 08:25:15.300405 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:25:15.300406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:25:15.300432 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:25:15.300513 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:25:15.300749 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.301530 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.302672 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:15.302969 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:15.303034 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:15.303261 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:25:15.303422 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:25:15.303514 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:25:15.303765 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 08:25:15.303788 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:25:15.313370 5159 raft_consensus.cc:3037] T 092dd5796e234ebca7327e354f74b926 P 1f73d6d74643494c92b706b2d89e0e27 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:25:15.314563 5159 raft_consensus.cc:1261] T 092dd5796e234ebca7327e354f74b926 P 1f73d6d74643494c92b706b2d89e0e27 [term 1 FOLLOWER]: Refusing update from remote peer dcc67942d65044e789ca5675e4bbf61c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:25:15.315088 5369 consensus_queue.cc:1035] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f73d6d74643494c92b706b2d89e0e27" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:25:15.317901 5021 raft_consensus.cc:1261] T 092dd5796e234ebca7327e354f74b926 P a73ab8d027bc4a268948b419aead9f02 [term 1 FOLLOWER]: Refusing update from remote peer dcc67942d65044e789ca5675e4bbf61c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:25:15.318329 5369 consensus_queue.cc:1035] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a73ab8d027bc4a268948b419aead9f02" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 45405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:25:15.321923 5375 mvcc.cc:205] Tried to move back new op lower bound from 7298488382720630784 to 7298488382497533952. Current Snapshot: MvccSnapshot[applied={T|T < 7298488382720630784}]
I20260619 08:25:15.324254 5376 mvcc.cc:205] Tried to move back new op lower bound from 7298488382720630784 to 7298488382497533952. Current Snapshot: MvccSnapshot[applied={T|T < 7298488382720630784}]
I20260619 08:25:15.326607 4855 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:25:15.326713 4855 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:25:15.327662 4855 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b9a4e9a4f76545509307f74124596708: Sending DeleteTablet for 3 replicas of tablet 092dd5796e234ebca7327e354f74b926
I20260619 08:25:15.328236 5140 tablet_service.cc:1514] Processing DeleteTablet for tablet 092dd5796e234ebca7327e354f74b926 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:25:15 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:33010
I20260619 08:25:15.328464 5001 tablet_service.cc:1514] Processing DeleteTablet for tablet 092dd5796e234ebca7327e354f74b926 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:25:15 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:46574
I20260619 08:25:15.329015 5276 tablet_service.cc:1514] Processing DeleteTablet for tablet 092dd5796e234ebca7327e354f74b926 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:25:15 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:55064
I20260619 08:25:15.329592 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 4936
W20260619 08:25:15.336023 4840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.127.129:45405 (error 108)
I20260619 08:25:15.336174 5385 tablet_replica.cc:330] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c: stopping tablet replica
I20260619 08:25:15.336352 5385 raft_consensus.cc:2229] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:25:15.336555 5385 raft_consensus.cc:2258] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:25:15.337330 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 5073
I20260619 08:25:15.338351 5385 ts_tablet_manager.cc:1905] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:25:15.340198 5385 ts_tablet_manager.cc:1918] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:25:15.342800 5385 log.cc:1196] T 092dd5796e234ebca7327e354f74b926 P dcc67942d65044e789ca5675e4bbf61c: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/092dd5796e234ebca7327e354f74b926
I20260619 08:25:15.343149 5357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:15.343010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:45405 (local address 127.0.0.1:46578)
0619 08:25:15.343054 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:25:15.343099 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.129:45405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 08:25:15.343204 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 5212
I20260619 08:25:15.343247 5357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:15.343151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:45173 (local address 127.0.0.1:33022)
0619 08:25:15.343214 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 08:25:15.343234 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.130:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 08:25:15.343394 4840 catalog_manager.cc:4584] TS a73ab8d027bc4a268948b419aead9f02 (127.25.127.129:45405): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 092dd5796e234ebca7327e354f74b926: Network error: Client connection negotiation failed: client connection to 127.25.127.129:45405: connect: Connection refused (error 111)
I20260619 08:25:15.348932 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 4823
I20260619 08:25:15.354919 26110 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 08:25:15.700042 26110 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 08:25:15.700206 26110 mini_ranger.cc:66] Stopping Ranger...
I20260619 08:25:16.045200 26110 mini_ranger.cc:68] Stopped Ranger
2026-06-19 08:25:16.045 UTC [4409] LOG: received smart shutdown request
2026-06-19 08:25:16.049 UTC [4409] LOG: background worker "logical replication launcher" (PID 4418) exited with exit code 1
2026-06-19 08:25:16.049 UTC [4413] LOG: shutting down
2026-06-19 08:25:16.214 UTC [4409] LOG: database system is shut down
2026-06-19T08:25:16Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48134 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:25:16 dist-test-slave-5rhg krb5kdc[5395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:25:16 dist-test-slave-5rhg krb5kdc[5395](info): set up 2 sockets
Jun 19 08:25:16 dist-test-slave-5rhg krb5kdc[5395](info): commencing operation
krb5kdc: starting...
W20260619 08:25:17.385038 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:25:17 dist-test-slave-5rhg krb5kdc[5395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:25:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:25:17Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.25.127.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.25.127.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.25.127.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/rangeradmin_127.25.127.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.25.127.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/rangeradmin_127.25.127.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.25.127.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.25.127.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.25.127.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/rangerlookup_127.25.127.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.25.127.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/rangerlookup_127.25.127.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.127.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.25.127.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.127.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/HTTP_127.25.127.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.127.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/HTTP_127.25.127.148@KRBTEST.COM.keytab.
I20260619 08:25:17.555013 26110 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task2BFFaB/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/postgres -l logfile start
2026-06-19 08:25:19.967 UTC [5425] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 08:25:19.968 UTC [5425] LOG: listening on IPv4 address "127.25.127.148", port 46871
2026-06-19 08:25:19.971 UTC [5425] LOG: listening on Unix socket "/tmp/.s.PGSQL.46871"
2026-06-19 08:25:20.006 UTC [5428] LOG: database system was shut down at 2026-06-19 08:25:18 UTC
2026-06-19 08:25:20.010 UTC [5425] LOG: database system is ready to accept connections
I20260619 08:25:20.956899 26110 mini_postgres.cc:96] Postgres bound to 46871
2026-06-19 08:25:20.963 UTC [5436] FATAL: database "slave" does not exist
127.25.127.148:46871 - accepting connections
I20260619 08:25:20.963752 26110 mini_ranger.cc:161] Starting Ranger...
I20260619 08:25:20.979558 26110 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 08:25:21.449767 26110 mini_ranger.cc:87] Created ranger Postgres database
I20260619 08:25:21.449858 26110 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-admin
2026-06-19 08:25:21,728 [I] DB FLAVOR :POSTGRES
2026-06-19 08:25:21,728 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 08:25:21,728 [I] Checking connection..
2026-06-19 08:25:21,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 08:25:21,969 [I] Checking connection passed.
2026-06-19 08:25:21,970 [I] --------- Verifying version history table ---------
2026-06-19 08:25:21,970 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 08:25:22,224 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 08:25:22,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 08:25:22,492 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 08:25:22,492 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 08:25:22,493 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 08:25:22.727 UTC [5501] WARNING: there is no transaction in progress
2026-06-19 08:25:22,734 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 08:25:22,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 08:25:22,976 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 08:25:22,976 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 08:25:22,976 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 08:25:23,223 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 08:25:23,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5rhg.c.gcp-upstream.internal','N') ;"
2026-06-19 08:25:23,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 08:25:23,947 [I] Table x_portal_user does not exist in database ranger
2026-06-19 08:25:23,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 08:25:24,195 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 08:25:24,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 08:25:24,432 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 08:25:24,684 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 08:25:24,685 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 08:25:25.648 UTC [5610] WARNING: there is no transaction in progress
2026-06-19 08:25:25.664 UTC [5610] WARNING: there is no transaction in progress
2026-06-19 08:25:25.680 UTC [5610] WARNING: there is no transaction in progress
2026-06-19 08:25:25.695 UTC [5610] WARNING: there is no transaction in progress
2026-06-19 08:25:25.711 UTC [5610] WARNING: there is no transaction in progress
2026-06-19 08:25:25.729 UTC [5610] WARNING: there is no transaction in progress
2026-06-19 08:25:25.785 UTC [5610] WARNING: there is no transaction in progress
2026-06-19 08:25:25.792 UTC [5610] WARNING: there is no transaction in progress
2026-06-19 08:25:25.805 UTC [5610] WARNING: there is no transaction in progress
2026-06-19 08:25:25.815 UTC [5610] WARNING: there is no transaction in progress
2026-06-19 08:25:25.825 UTC [5610] WARNING: there is no transaction in progress
2026-06-19 08:25:25.840 UTC [5610] WARNING: there is no transaction in progress
2026-06-19 08:25:25.850 UTC [5610] WARNING: there is no transaction in progress
2026-06-19 08:25:25.856 UTC [5610] WARNING: there is no transaction in progress
2026-06-19 08:25:25.862 UTC [5610] WARNING: there is no transaction in progress
2026-06-19 08:25:25.869 UTC [5610] WARNING: there is no transaction in progress
2026-06-19 08:25:26,510 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 08:25:26,510 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 08:25:26,780 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 08:25:26,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 08:25:27,051 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 08:25:27,051 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 08:25:27,315 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 08:25:27,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 08:25:27,589 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 08:25:27,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5rhg.c.gcp-upstream.internal';"
2026-06-19 08:25:28,060 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 08:25:28,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 08:25:28,320 [I] DB_PATCHES have already been applied
I20260619 08:25:28.326064 26110 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-admin:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task2BFFaB/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 08:25:28.326135 26110 mini_ranger.cc:193] Using host: 127.25.127.148
I20260619 08:25:28.329329 26110 mini_ranger.cc:227] Ranger admin URL: http://127.25.127.148:52675
Jun 19, 2026 8:25:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:25:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:25:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 8:25:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 8:25:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 8:25:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 8:25:29 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 08:25:29.435443 26110 mini_ranger.cc:160] Time spent starting Ranger: real 8.472s user 0.000s sys 0.009s
Jun 19, 2026 8:25:49 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 8:25:49 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Jun 19, 2026 8:25:49 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 8:25:49 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 8:25:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 8:25:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 8:25:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 8:25:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 8:25:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 8:25:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 8:25:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 8:25:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 8:25:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 8:25:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 8:25:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 8:25:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 8:25:49 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 8:25:50 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 08:25:51.188107 26110 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.25.127.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.25.127.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.25.127.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/rangerkms_127.25.127.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.25.127.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/rangerkms_127.25.127.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.127.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.25.127.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.127.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/HTTP_127.25.127.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.127.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/HTTP_127.25.127.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 08:25:51 dist-test-slave-5rhg krb5kdc[5395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857551, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 08:25:51.270385 26110 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 08:25:51.279991 26110 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 08:25:52.083979 26110 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 08:25:52.084048 26110 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-kms
I20260619 08:25:52.084061 26110 mini_ranger_kms.cc:227] Using postgres at 127.25.127.148:46871
2026-06-19 08:25:52,249 [I] DB FLAVOR :POSTGRES
2026-06-19 08:25:52,249 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 08:25:52,249 [I] Checking connection
2026-06-19 08:25:52,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 08:25:52,483 [I] connection success
2026-06-19 08:25:52,483 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 08:25:52,483 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 08:25:52,483 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 08:25:52,719 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 08:25:52,719 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 08:25:52,719 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 08:25:52,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.127.148:46871/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 08:25:52,990 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 08:25:53.518397 26110 mini_ranger_kms.cc:310] Created kms service
I20260619 08:25:53.737010 26110 mini_ranger_kms.cc:326] Created kudu user
I20260619 08:25:53.786183 26110 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 08:25:53 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857551, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
I20260619 08:25:54.230902 26110 mini_ranger_kms.cc:384] Added ranger policy
I20260619 08:25:54.231118 26110 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-kms:/tmp/dist-test-task2BFFaB/build/release/bin/postgresql.jar:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task2BFFaB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 08:25:54.231140 26110 mini_ranger_kms.cc:240] Using host: 127.25.127.148
I20260619 08:25:54.235074 26110 mini_ranger_kms.cc:276] Ranger KMS PID: 5791
I20260619 08:25:54.235157 26110 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.25.127.148:44351
Jun 19, 2026 8:25:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:25:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:25:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 8:25:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 8:25:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 8:25:55 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 08:25:56.447630 26110 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.177s user 0.001s sys 0.147s
I20260619 08:25:56.447742 26110 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 08:25:56.447795 26110 mini_ranger_kms.cc:401] 127.25.127.148:44351/kms/v1/keys
Jun 19 08:25:56 dist-test-slave-5rhg krb5kdc[5395](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781857556, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.127.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 08:25:57 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857556, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.127.148@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
Jun 19 08:25:57 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857556, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.127.148@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
Jun 19 08:25:59 dist-test-slave-5rhg krb5kdc[5395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:25:59.706528 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:46035
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:46035
--ranger_config_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.127.148:44351/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:25:59.788128 5842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:25:59.788405 5842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:25:59.788429 5842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:25:59.789646 5842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:25:59.789690 5842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:25:59.789705 5842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:25:59.789717 5842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:25:59.789731 5842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:25:59.791450 5842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/wal
--ranger_kms_url=127.25.127.148:44351/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:46035
--ranger_config_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:46035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:25:59.791703 5842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:25:59.791957 5842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:25:59.797416 5842 server_base.cc:1034] running on GCE node
W20260619 08:25:59.797374 5847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:25:59.797390 5848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:25:59.797600 5850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:25:59.797874 5842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:25:59.798211 5842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:25:59.799368 5842 hybrid_clock.cc:648] HybridClock initialized: now 1781857559799336 us; error 40 us; skew 500 ppm
Jun 19 08:25:59 dist-test-slave-5rhg krb5kdc[5395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857559, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:25:59.802281 5842 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:25:59.802763 5842 webserver.cc:460] Webserver started at http://127.25.127.190:38267/ using document root <none> and password file <none>
I20260619 08:25:59.802996 5842 fs_manager.cc:359] Metadata directory not provided
I20260619 08:25:59.803061 5842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:25:59.803218 5842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:25:59 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857559, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
I20260619 08:25:59.862541 5842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "943dfa643151426eae072de881a05f2d"
format_stamp: "Formatted at 2026-06-19 08:25:59 on dist-test-slave-5rhg"
server_key: "53fde8fd6c9b65f3c6a2b0baac265d5f"
server_key_iv: "5533060f0a940af4b111236d3012198f"
server_key_version: "kuduclusterkey@0"
I20260619 08:25:59.862946 5842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "943dfa643151426eae072de881a05f2d"
format_stamp: "Formatted at 2026-06-19 08:25:59 on dist-test-slave-5rhg"
server_key: "53fde8fd6c9b65f3c6a2b0baac265d5f"
server_key_iv: "5533060f0a940af4b111236d3012198f"
server_key_version: "kuduclusterkey@0"
I20260619 08:25:59.864437 5842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:25:59.879916 5857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:25:59.880192 5842 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:25:59.880335 5842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "943dfa643151426eae072de881a05f2d"
format_stamp: "Formatted at 2026-06-19 08:25:59 on dist-test-slave-5rhg"
server_key: "53fde8fd6c9b65f3c6a2b0baac265d5f"
server_key_iv: "5533060f0a940af4b111236d3012198f"
server_key_version: "kuduclusterkey@0"
I20260619 08:25:59.880445 5842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:25:59.898232 5842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:25:59.900470 5842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:25:59.900640 5842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:25:59.905568 5842 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:46035
I20260619 08:25:59.905596 5910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:46035 every 8 connection(s)
I20260619 08:25:59.906018 5842 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:25:59.906718 5911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:25:59.909052 5911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d: Bootstrap starting.
I20260619 08:25:59.909654 5911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d: Neither blocks nor log segments found. Creating new log.
I20260619 08:25:59.909952 5911 log.cc:824] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d: Log is configured to *not* fsync() on all Append() calls
I20260619 08:25:59.910648 5911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d: No bootstrap required, opened a new log
I20260619 08:25:59.912508 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 5842
I20260619 08:25:59.912604 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:25:59.912784 26110 external_mini_cluster.cc:1428] Setting key 79d7c2d746b14fd9ec889a90860c7775
I20260619 08:25:59.913086 5911 raft_consensus.cc:348] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943dfa643151426eae072de881a05f2d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46035 } }
I20260619 08:25:59.913295 5911 raft_consensus.cc:374] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:25:59.913336 5911 raft_consensus.cc:729] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 943dfa643151426eae072de881a05f2d, State: Initialized, Role: FOLLOWER
I20260619 08:25:59.913511 5911 consensus_queue.cc:260] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943dfa643151426eae072de881a05f2d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46035 } }
I20260619 08:25:59.913707 5911 raft_consensus.cc:388] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:25:59.913775 5911 raft_consensus.cc:482] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:25:59.913858 5911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:25:59.914675 5911 raft_consensus.cc:504] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943dfa643151426eae072de881a05f2d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46035 } }
I20260619 08:25:59.914799 5911 leader_election.cc:302] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 943dfa643151426eae072de881a05f2d; no voters:
I20260619 08:25:59.914974 5911 leader_election.cc:288] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:25:59.915144 5917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:25:59.915257 5911 sys_catalog.cc:564] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:25:59.915320 5917 raft_consensus.cc:686] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [term 1 LEADER]: Becoming Leader. State: Replica: 943dfa643151426eae072de881a05f2d, State: Running, Role: LEADER
I20260619 08:25:59.915413 5917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943dfa643151426eae072de881a05f2d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46035 } }
I20260619 08:25:59.915729 5917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "943dfa643151426eae072de881a05f2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943dfa643151426eae072de881a05f2d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46035 } } }
I20260619 08:25:59.915809 5917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [sys.catalog]: This master's current role is: LEADER
I20260619 08:25:59.915938 5917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 943dfa643151426eae072de881a05f2d. Latest consensus state: current_term: 1 leader_uuid: "943dfa643151426eae072de881a05f2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943dfa643151426eae072de881a05f2d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46035 } } }
I20260619 08:25:59.916013 5917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [sys.catalog]: This master's current role is: LEADER
I20260619 08:25:59.917048 5911 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 08:25:59.919461 5920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:25:59.920763 5920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 08:25:59 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:25:59.926457 5920 catalog_manager.cc:1347] Generated new cluster ID: ae9af3d0a1e34c00a141d1cbee3ee787
I20260619 08:25:59.926512 5920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:25:59.928315 5915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:25:59.913502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55682 (local address 127.25.127.190:46035)
0619 08:25:59.913752 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0619 08:25:59.913762 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:25:59.913799 (+ 37us) server_negotiation.cc:407] Connection header received
0619 08:25:59.913923 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:25:59.913931 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:25:59.913988 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:25:59.914138 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:25:59.914990 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:59.916533 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:59.918821 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:25:59.920533 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:25:59.921024 (+ 491us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:25:59.923674 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:25:59.923678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:25:59.923678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:25:59.923690 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:25:59.926073 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:25:59.926518 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:25:59.926522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:25:59.926522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:25:59.926651 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:25:59.926865 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:25:59.926865 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:25:59.926866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:25:59.927181 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:25:59.927303 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 08:25:59.927923 (+ 620us) server_negotiation.cc:299] Negotiation successful
0619 08:25:59.928056 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":82,"threads_started":1}
I20260619 08:25:59.939102 5920 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:25:59.939949 5920 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:25:59.942365 5920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d: Generated new TSK 0
I20260619 08:25:59.942613 5920 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
I20260619 08:26:00.763160 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.127.148:44351/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:46035
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:26:00.879710 5956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:00.880000 5956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:00.880050 5956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:00.881949 5956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:00.882028 5956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:00.882073 5956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:26:00.884637 5956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.25.127.148:44351/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:46035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:00.885110 5956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:00.885413 5956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:00.891847 5963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:00.891835 5964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:00.891880 5966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:00.892356 5956 server_base.cc:1034] running on GCE node
I20260619 08:26:00.892580 5956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:00.892884 5956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:00.894062 5956 hybrid_clock.cc:648] HybridClock initialized: now 1781857560894042 us; error 36 us; skew 500 ppm
Jun 19 08:26:00 dist-test-slave-5rhg krb5kdc[5395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857560, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:00.897047 5956 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:26:00.897532 5956 webserver.cc:460] Webserver started at http://127.25.127.129:45911/ using document root <none> and password file <none>
I20260619 08:26:00.897797 5956 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:00.897873 5956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:00.898030 5956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:26:00 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857560, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
I20260619 08:26:00.918304 5956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "1cab943d9fe04d65bb1772bba20458f4"
format_stamp: "Formatted at 2026-06-19 08:26:00 on dist-test-slave-5rhg"
server_key: "47297a32cdb3eb889fb0695873e87109"
server_key_iv: "ae475d7e387487a93720949b1ee5d488"
server_key_version: "kuduclusterkey@0"
I20260619 08:26:00.920498 5956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "1cab943d9fe04d65bb1772bba20458f4"
format_stamp: "Formatted at 2026-06-19 08:26:00 on dist-test-slave-5rhg"
server_key: "47297a32cdb3eb889fb0695873e87109"
server_key_iv: "ae475d7e387487a93720949b1ee5d488"
server_key_version: "kuduclusterkey@0"
I20260619 08:26:00.928687 5956 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.001s sys 0.000s
I20260619 08:26:00.942638 5973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:00.943022 5956 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:00.943140 5956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "1cab943d9fe04d65bb1772bba20458f4"
format_stamp: "Formatted at 2026-06-19 08:26:00 on dist-test-slave-5rhg"
server_key: "47297a32cdb3eb889fb0695873e87109"
server_key_iv: "ae475d7e387487a93720949b1ee5d488"
server_key_version: "kuduclusterkey@0"
I20260619 08:26:00.943241 5956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:00.966315 5956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:00.969015 5956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:00.969238 5956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:00.969563 5956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:00.970002 5956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:26:00.970100 5956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:00.970196 5956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:26:00.970274 5956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:00.977555 5956 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:33971
I20260619 08:26:00.978123 5956 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:26:00.980762 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 5956
I20260619 08:26:00.980867 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:26:00.981014 6087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:33971 every 8 connection(s)
I20260619 08:26:00.981076 26110 external_mini_cluster.cc:1428] Setting key 6d035018e799c1a2b59a437259c25b23
Jun 19 08:26:00 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857560, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.127.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.130@KRBTEST.COM" created.
I20260619 08:26:00.996124 6090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:00.980550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46035 (local address 127.25.127.129:40867)
0619 08:26:00.982237 (+ 1687us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:00.982290 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:00.983063 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:00.985608 (+ 2545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:00.985615 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:00.985926 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:00.986220 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:00.986226 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:00.987280 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:00.987281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:00.988428 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:00.988431 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:00.988907 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:00.988909 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:00.989095 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:00.989837 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:00.989850 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:00.991901 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:00.994228 (+ 2327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:00.994237 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:00.994239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:00.994541 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:00.994762 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:00.994762 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:00.994763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:00.994982 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:00.995307 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:00.995313 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:00.995522 (+ 209us) client_negotiation.cc:769] Sending connection context
0619 08:26:00.995641 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 08:26:00.995821 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1634,"thread_start_us":91,"threads_started":1}
I20260619 08:26:00.996820 6091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:00.980856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:40867 (local address 127.25.127.190:46035)
0619 08:26:00.983622 (+ 2766us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:00.983624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:00.983634 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:26:00.983660 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:00.983661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:00.983674 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:00.983745 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:00.986340 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:00.987147 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:00.988557 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:00.988783 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:00.988854 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:00.992043 (+ 3189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:00.992045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:00.992046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:00.992054 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:00.994110 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:00.994630 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:00.994631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:00.994631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:00.994696 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:00.995065 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:00.995067 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:00.995067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:00.995215 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:00.995279 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:00.996570 (+ 1291us) server_negotiation.cc:299] Negotiation successful
0619 08:26:00.996672 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2739,"thread_start_us":67,"threads_started":1}
I20260619 08:26:00.997320 6088 heartbeater.cc:347] Connected to a master server at 127.25.127.190:46035
I20260619 08:26:00.997416 6088 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:00.997604 6088 heartbeater.cc:511] Master 127.25.127.190:46035 requested a full tablet report, sending...
I20260619 08:26:00.998167 5874 ts_manager.cc:194] Registered new tserver with Master: 1cab943d9fe04d65bb1772bba20458f4 (127.25.127.129:33971)
I20260619 08:26:00.999042 5874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:40867
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
I20260619 08:26:01.023590 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.127.148:44351/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:46035
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 08:26:01 dist-test-slave-5rhg krb5kdc[5395](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781857561, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 08:26:01.144083 6096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:01.144416 6096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:01.144456 6096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:01.146180 6096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:01.146245 6096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:01.146284 6096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:26:01.148447 6096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.25.127.148:44351/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:46035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:01.148717 6096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:01.149058 6096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:01.155426 6104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:01.155640 6101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:01.156381 6102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:01.156639 6096 server_base.cc:1034] running on GCE node
I20260619 08:26:01.157081 6096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:01.157392 6096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:01.158583 6096 hybrid_clock.cc:648] HybridClock initialized: now 1781857561158558 us; error 41 us; skew 500 ppm
Jun 19 08:26:01 dist-test-slave-5rhg krb5kdc[5395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857561, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:01.161600 6096 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:26:01.162158 6096 webserver.cc:460] Webserver started at http://127.25.127.130:38473/ using document root <none> and password file <none>
I20260619 08:26:01.162425 6096 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:01.162492 6096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:01.162675 6096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:26:01 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857561, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
I20260619 08:26:01.181854 6096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "0b58ced57a8244b791103f7e3b45a7f7"
format_stamp: "Formatted at 2026-06-19 08:26:01 on dist-test-slave-5rhg"
server_key: "43d9e2c7e8e0511b1f1f93bb02715322"
server_key_iv: "e789b69509b7bd1b6408ebd75fbb7962"
server_key_version: "kuduclusterkey@0"
I20260619 08:26:01.182322 6096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "0b58ced57a8244b791103f7e3b45a7f7"
format_stamp: "Formatted at 2026-06-19 08:26:01 on dist-test-slave-5rhg"
server_key: "43d9e2c7e8e0511b1f1f93bb02715322"
server_key_iv: "e789b69509b7bd1b6408ebd75fbb7962"
server_key_version: "kuduclusterkey@0"
I20260619 08:26:01.184271 6096 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260619 08:26:01.201754 6112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:01.201951 6096 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:01.202060 6096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "0b58ced57a8244b791103f7e3b45a7f7"
format_stamp: "Formatted at 2026-06-19 08:26:01 on dist-test-slave-5rhg"
server_key: "43d9e2c7e8e0511b1f1f93bb02715322"
server_key_iv: "e789b69509b7bd1b6408ebd75fbb7962"
server_key_version: "kuduclusterkey@0"
I20260619 08:26:01.202165 6096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:01.240983 6096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:01.243321 6096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:01.243495 6096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:01.243821 6096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:01.244302 6096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:26:01.244396 6096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:01.244455 6096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:26:01.244499 6096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:01.251668 6096 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:45149
I20260619 08:26:01.251816 6225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:45149 every 8 connection(s)
I20260619 08:26:01.252203 6096 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:26:01.256819 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 6096
I20260619 08:26:01.256949 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:26:01.257145 26110 external_mini_cluster.cc:1428] Setting key 69f3c8edc2ca7b313535b991285b7908
Jun 19 08:26:01 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857561, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.127.131@KRBTEST.COM; defaulting to no policy
I20260619 08:26:01.268047 6091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:01.253285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:33569 (local address 127.25.127.190:46035)
0619 08:26:01.253336 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:01.253337 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:01.253580 (+ 243us) server_negotiation.cc:407] Connection header received
0619 08:26:01.254307 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:01.254308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:01.254328 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:01.254376 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:01.255249 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:01.256060 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:01.257876 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:01.258195 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:01.258242 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:01.264375 (+ 6133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:01.264377 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:01.264377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:01.264387 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:01.266278 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:01.266854 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:01.266855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:01.266856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:01.266918 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:01.267201 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:01.267202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:01.267202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:01.267346 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:01.267392 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:01.267816 (+ 424us) server_negotiation.cc:299] Negotiation successful
0619 08:26:01.267916 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:26:01.268647 6228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:01.253260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46035 (local address 127.25.127.130:33569)
0619 08:26:01.253438 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:01.253488 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:01.254171 (+ 683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:01.254582 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:01.254587 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:01.254837 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:01.255129 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:01.255135 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:01.256225 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:01.256226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:01.257762 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:01.257765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:01.258359 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:01.258360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:01.258503 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:01.259162 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:01.259173 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:01.260905 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:01.266393 (+ 5488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:01.266404 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:01.266408 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:01.266758 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:01.266990 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:01.266990 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:01.266991 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:01.267129 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:01.267458 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:01.267465 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:01.267652 (+ 187us) client_negotiation.cc:769] Sending connection context
0619 08:26:01.268186 (+ 534us) client_negotiation.cc:240] Negotiation successful
0619 08:26:01.268370 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260619 08:26:01.269196 6226 heartbeater.cc:347] Connected to a master server at 127.25.127.190:46035
I20260619 08:26:01.269320 6226 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:01.269554 6226 heartbeater.cc:511] Master 127.25.127.190:46035 requested a full tablet report, sending...
I20260619 08:26:01.269963 5874 ts_manager.cc:194] Registered new tserver with Master: 0b58ced57a8244b791103f7e3b45a7f7 (127.25.127.130:45149)
I20260619 08:26:01.270488 5874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:33569
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
I20260619 08:26:01.290645 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.127.148:44351/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:46035
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:26:01.411267 6231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:01.411653 6231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:01.411749 6231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:01.413715 6231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:01.413779 6231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:01.413821 6231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:26:01.416208 6231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.25.127.148:44351/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:46035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:01.416498 6231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:01.416795 6231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:01.423745 6237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:01.424007 6239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:01.424430 6236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:01.425586 6231 server_base.cc:1034] running on GCE node
I20260619 08:26:01.425851 6231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:01.426154 6231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:01.427331 6231 hybrid_clock.cc:648] HybridClock initialized: now 1781857561427301 us; error 46 us; skew 500 ppm
Jun 19 08:26:01 dist-test-slave-5rhg krb5kdc[5395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857561, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:01.430927 6231 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:26:01.431478 6231 webserver.cc:460] Webserver started at http://127.25.127.131:44855/ using document root <none> and password file <none>
I20260619 08:26:01.431754 6231 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:01.431814 6231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:01.431995 6231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:26:01 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857561, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
I20260619 08:26:01.448189 6231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "d6cef17caf934179bfc9205a06befc1a"
format_stamp: "Formatted at 2026-06-19 08:26:01 on dist-test-slave-5rhg"
server_key: "cb795744aead814ed7378ad37f83110d"
server_key_iv: "8a2706b180b21383ef86a8c085cd3e07"
server_key_version: "kuduclusterkey@0"
I20260619 08:26:01.448699 6231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "d6cef17caf934179bfc9205a06befc1a"
format_stamp: "Formatted at 2026-06-19 08:26:01 on dist-test-slave-5rhg"
server_key: "cb795744aead814ed7378ad37f83110d"
server_key_iv: "8a2706b180b21383ef86a8c085cd3e07"
server_key_version: "kuduclusterkey@0"
I20260619 08:26:01.460004 6231 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.001s sys 0.000s
I20260619 08:26:01.480795 6246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:01.481104 6231 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260619 08:26:01.481215 6231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "d6cef17caf934179bfc9205a06befc1a"
format_stamp: "Formatted at 2026-06-19 08:26:01 on dist-test-slave-5rhg"
server_key: "cb795744aead814ed7378ad37f83110d"
server_key_iv: "8a2706b180b21383ef86a8c085cd3e07"
server_key_version: "kuduclusterkey@0"
I20260619 08:26:01.481320 6231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:01.500620 6231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:01.503217 6231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:01.503374 6231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:01.503641 6231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:01.504074 6231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:26:01.504127 6231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:01.504189 6231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:26:01.504248 6231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:01.511430 6231 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:46299
I20260619 08:26:01.511976 6231 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:26:01.512555 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 6231
I20260619 08:26:01.512667 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:26:01.512871 26110 external_mini_cluster.cc:1428] Setting key e1537d6e8487ab64fd1da0f955a93b27
I20260619 08:26:01.513070 6359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:46299 every 8 connection(s)
Jun 19 08:26:01 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857561, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:01.525012 6091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:01.513998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:36347 (local address 127.25.127.190:46035)
0619 08:26:01.514037 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:01.514039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:01.514067 (+ 28us) server_negotiation.cc:407] Connection header received
0619 08:26:01.515044 (+ 977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:01.515046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:01.515059 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:01.515127 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:01.515873 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:01.516715 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:01.518280 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:01.518586 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:01.518665 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:01.521356 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:01.521357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:01.521357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:01.521365 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:01.523198 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:01.523745 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:01.523747 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:01.523747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:01.523815 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:01.524225 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:01.524226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:01.524227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:01.524374 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:01.524424 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:01.524833 (+ 409us) server_negotiation.cc:299] Negotiation successful
0619 08:26:01.524913 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:26:01.525205 6362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:01.513704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46035 (local address 127.25.127.131:36347)
0619 08:26:01.513923 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:01.513975 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:01.514856 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:01.515204 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:01.515209 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:01.515466 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:01.515749 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:01.515756 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:01.516827 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:01.516829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:01.518184 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:01.518187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:01.518704 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:01.518705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:01.518851 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:01.519489 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:01.519500 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:01.521245 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:01.523290 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:01.523336 (+ 46us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:01.523339 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:01.523649 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:01.523934 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:01.523935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:01.523935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:01.524083 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:01.524477 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:01.524484 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:01.524653 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 08:26:01.524794 (+ 141us) client_negotiation.cc:240] Negotiation successful
0619 08:26:01.524970 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":84,"threads_started":1}
I20260619 08:26:01.525794 6360 heartbeater.cc:347] Connected to a master server at 127.25.127.190:46035
I20260619 08:26:01.525902 6360 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:01.526101 6360 heartbeater.cc:511] Master 127.25.127.190:46035 requested a full tablet report, sending...
I20260619 08:26:01.526459 5874 ts_manager.cc:194] Registered new tserver with Master: d6cef17caf934179bfc9205a06befc1a (127.25.127.131:46299)
I20260619 08:26:01.526930 5874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:36347
I20260619 08:26:01.535282 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:26:01.535964 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 5956
I20260619 08:26:01.543792 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 6096
I20260619 08:26:01.549963 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 6231
Jun 19 08:26:01 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857561, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
I20260619 08:26:01.566015 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 5842
I20260619 08:26:01.572077 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:46035
--webserver_interface=127.25.127.190
--webserver_port=38267
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:46035
--ranger_config_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.127.148:44351/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.127.148:44351/kms with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:26:01.656556 6363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:01.656774 6363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:01.656798 6363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:01.657925 6363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:26:01.657971 6363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:01.657989 6363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:01.658002 6363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:26:01.658016 6363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:26:01.659529 6363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.127.148:44351/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:46035
--ranger_config_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:46035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=38267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:01.659716 6363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:01.659907 6363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:01.664959 6368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:01.665112 6363 server_base.cc:1034] running on GCE node
W20260619 08:26:01.665233 6369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:01.665282 6371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:01.665593 6363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:01.665864 6363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:01.667011 6363 hybrid_clock.cc:648] HybridClock initialized: now 1781857561667011 us; error 42 us; skew 500 ppm
Jun 19 08:26:01 dist-test-slave-5rhg krb5kdc[5395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857561, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:01.669687 6363 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:26:01.670073 6363 webserver.cc:460] Webserver started at http://127.25.127.190:38267/ using document root <none> and password file <none>
I20260619 08:26:01.670261 6363 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:01.670310 6363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 08:26:01 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857561, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
I20260619 08:26:01.710956 6363 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:01.711938 6379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:01.712186 6363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:01.712288 6363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "943dfa643151426eae072de881a05f2d"
format_stamp: "Formatted at 2026-06-19 08:25:59 on dist-test-slave-5rhg"
server_key: "53fde8fd6c9b65f3c6a2b0baac265d5f"
server_key_iv: "5533060f0a940af4b111236d3012198f"
server_key_version: "kuduclusterkey@0"
I20260619 08:26:01.712703 6363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:01.731798 6363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:01.733675 6363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:01.733844 6363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:01.737973 6363 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:46035
I20260619 08:26:01.738023 6431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:46035 every 8 connection(s)
I20260619 08:26:01.738394 6363 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:26:01.739627 6432 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:26:01.740357 6432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d: Bootstrap starting.
I20260619 08:26:01.748268 6432 log.cc:824] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:01.748378 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 6363
I20260619 08:26:01.748683 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.127.148:44351/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:33971
--local_ip_for_outbound_sockets=127.25.127.129
--tserver_master_addrs=127.25.127.190:46035
--webserver_port=45911
--webserver_interface=127.25.127.129
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.127.148:44351/kms with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:26:01.750079 6432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:26:01.750360 6432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d: Bootstrap complete.
I20260619 08:26:01.754403 6432 raft_consensus.cc:348] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943dfa643151426eae072de881a05f2d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46035 } }
I20260619 08:26:01.754796 6432 raft_consensus.cc:729] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 943dfa643151426eae072de881a05f2d, State: Initialized, Role: FOLLOWER
I20260619 08:26:01.754993 6432 consensus_queue.cc:260] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943dfa643151426eae072de881a05f2d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46035 } }
I20260619 08:26:01.755082 6432 raft_consensus.cc:388] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:26:01.755131 6432 raft_consensus.cc:482] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:26:01.755190 6432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:26:01.756844 6432 raft_consensus.cc:504] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943dfa643151426eae072de881a05f2d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46035 } }
I20260619 08:26:01.756986 6432 leader_election.cc:302] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 943dfa643151426eae072de881a05f2d; no voters:
I20260619 08:26:01.757179 6432 leader_election.cc:288] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 08:26:01.757251 6436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [term 2 FOLLOWER]: Leader election won for term 2
I20260619 08:26:01.757447 6436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [term 2 LEADER]: Becoming Leader. State: Replica: 943dfa643151426eae072de881a05f2d, State: Running, Role: LEADER
I20260619 08:26:01.757468 6432 sys_catalog.cc:564] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:26:01.757616 6436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943dfa643151426eae072de881a05f2d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46035 } }
I20260619 08:26:01.757992 6437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "943dfa643151426eae072de881a05f2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943dfa643151426eae072de881a05f2d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46035 } } }
I20260619 08:26:01.758066 6437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [sys.catalog]: This master's current role is: LEADER
I20260619 08:26:01.758307 6438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 943dfa643151426eae072de881a05f2d. Latest consensus state: current_term: 2 leader_uuid: "943dfa643151426eae072de881a05f2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943dfa643151426eae072de881a05f2d" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46035 } } }
I20260619 08:26:01.758378 6438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d [sys.catalog]: This master's current role is: LEADER
I20260619 08:26:01.758533 6442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:26:01.759088 6442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:26:01.759254 6442 catalog_manager.cc:1259] Loaded cluster ID: ae9af3d0a1e34c00a141d1cbee3ee787
I20260619 08:26:01.759325 6442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:26:01.759428 6432 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 08:26:01.763430 6442 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:26:01.763849 6442 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d: Loaded TSK: 0
I20260619 08:26:01.764196 6442 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 08:26:01.865527 6435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:01.865923 6435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:01.866039 6435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:01.867909 6435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:01.868047 6435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:01.868170 6435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:26:01.870635 6435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.127.148:44351/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:33971
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=45911
--tserver_master_addrs=127.25.127.190:46035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:01.871228 6435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:01.871605 6435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:01.878063 6473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:01.878305 6471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:01.878060 6470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:01.879341 6435 server_base.cc:1034] running on GCE node
I20260619 08:26:01.879565 6435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:01.879858 6435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:01.881034 6435 hybrid_clock.cc:648] HybridClock initialized: now 1781857561880722 us; error 327 us; skew 500 ppm
Jun 19 08:26:01 dist-test-slave-5rhg krb5kdc[5395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857561, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:01.891709 6435 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:26:01.892418 6435 webserver.cc:460] Webserver started at http://127.25.127.129:45911/ using document root <none> and password file <none>
I20260619 08:26:01.892772 6435 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:01.892897 6435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 08:26:01 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857561, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
I20260619 08:26:01.915987 6435 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.001s sys 0.000s
I20260619 08:26:01.919867 6481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:01.920120 6435 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260619 08:26:01.920234 6435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "1cab943d9fe04d65bb1772bba20458f4"
format_stamp: "Formatted at 2026-06-19 08:26:00 on dist-test-slave-5rhg"
server_key: "47297a32cdb3eb889fb0695873e87109"
server_key_iv: "ae475d7e387487a93720949b1ee5d488"
server_key_version: "kuduclusterkey@0"
I20260619 08:26:01.920722 6435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:01.932129 6435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:01.934661 6435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:01.934875 6435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:01.935269 6435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:01.935684 6435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:26:01.935770 6435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:01.935868 6435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:26:01.935936 6435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:01.942531 6435 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:33971
I20260619 08:26:01.943050 6435 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:26:01.945461 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 6435
I20260619 08:26:01.945797 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.127.148:44351/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:45149
--local_ip_for_outbound_sockets=127.25.127.130
--tserver_master_addrs=127.25.127.190:46035
--webserver_port=38473
--webserver_interface=127.25.127.130
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.127.148:44351/kms with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:26:01.947567 6596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:33971 every 8 connection(s)
Jun 19 08:26:01 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857561, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:01.967981 6600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:01.948757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46035 (local address 127.25.127.129:51781)
0619 08:26:01.951763 (+ 3006us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:01.951828 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:01.952506 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:01.953144 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:01.953148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:01.953389 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:01.953660 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:01.953665 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:01.956215 (+ 2550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:01.956216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:01.957363 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:01.957367 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:01.960188 (+ 2821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:01.960190 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:01.960351 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:01.961078 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:01.961091 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:01.963328 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:01.965861 (+ 2533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:01.965869 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:01.965871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:01.966186 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:01.966488 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:01.966489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:01.966489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:01.966656 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:01.967237 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:01.967244 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:01.967410 (+ 166us) client_negotiation.cc:769] Sending connection context
0619 08:26:01.967522 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 08:26:01.967706 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2957,"thread_start_us":97,"threads_started":1}
I20260619 08:26:01.968940 6601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:01.948829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:51781 (local address 127.25.127.190:46035)
0619 08:26:01.950896 (+ 2067us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:01.950908 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:01.952677 (+ 1769us) server_negotiation.cc:407] Connection header received
0619 08:26:01.952830 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:01.952836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:01.952892 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:01.953008 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:01.953786 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:01.955190 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:01.959116 (+ 3926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:01.959859 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:01.960007 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:01.963494 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:01.963500 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:01.963500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:01.963515 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:01.965757 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:01.966289 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:01.966294 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:01.966295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:01.966403 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:01.966748 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:01.966749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:01.966750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:01.967114 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:01.967205 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:01.968549 (+ 1344us) server_negotiation.cc:299] Negotiation successful
0619 08:26:01.968705 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1955,"thread_start_us":98,"threads_started":1}
I20260619 08:26:01.970178 6597 heartbeater.cc:347] Connected to a master server at 127.25.127.190:46035
I20260619 08:26:01.970304 6597 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:01.970517 6597 heartbeater.cc:511] Master 127.25.127.190:46035 requested a full tablet report, sending...
W20260619 08:26:01.971467 6597 heartbeater.cc:644] Failed to heartbeat to 127.25.127.190:46035 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 08:26:01.971544 6597 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:01.971611 6597 heartbeater.cc:511] Master 127.25.127.190:46035 requested a full tablet report, sending...
I20260619 08:26:01.972033 6597 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:01.972111 6597 heartbeater.cc:511] Master 127.25.127.190:46035 requested a full tablet report, sending...
W20260619 08:26:01.972487 6597 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 08:26:02.063489 6599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:02.063807 6599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:02.063860 6599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:02.065425 6599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:02.065512 6599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:02.065558 6599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:26:02.067097 6599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.127.148:44351/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:45149
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=38473
--tserver_master_addrs=127.25.127.190:46035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:02.067389 6599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:02.067654 6599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:02.074158 6607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:02.074246 6609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:02.074486 6606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:02.074553 6599 server_base.cc:1034] running on GCE node
I20260619 08:26:02.074811 6599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:02.075111 6599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:02.076277 6599 hybrid_clock.cc:648] HybridClock initialized: now 1781857562076250 us; error 40 us; skew 500 ppm
Jun 19 08:26:02 dist-test-slave-5rhg krb5kdc[5395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857562, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:02.079397 6599 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:26:02.079945 6599 webserver.cc:460] Webserver started at http://127.25.127.130:38473/ using document root <none> and password file <none>
I20260619 08:26:02.080269 6599 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:02.080351 6599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 08:26:02 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857562, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
I20260619 08:26:02.101727 6599 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260619 08:26:02.102764 6617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:02.102974 6599 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:02.103071 6599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "0b58ced57a8244b791103f7e3b45a7f7"
format_stamp: "Formatted at 2026-06-19 08:26:01 on dist-test-slave-5rhg"
server_key: "43d9e2c7e8e0511b1f1f93bb02715322"
server_key_iv: "e789b69509b7bd1b6408ebd75fbb7962"
server_key_version: "kuduclusterkey@0"
I20260619 08:26:02.103474 6599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:02.114059 6599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:02.116580 6599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:02.116752 6599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:02.117089 6599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:02.117516 6599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:26:02.117607 6599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:02.117679 6599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:26:02.117736 6599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:02.125183 6599 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:45149
I20260619 08:26:02.125277 6731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:45149 every 8 connection(s)
I20260619 08:26:02.125648 6599 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:26:02 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857562, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:02.134282 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 6599
I20260619 08:26:02.134583 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.127.148:44351/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:46299
--local_ip_for_outbound_sockets=127.25.127.131
--tserver_master_addrs=127.25.127.190:46035
--webserver_port=44855
--webserver_interface=127.25.127.131
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.127.148:44351/kms with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:26:02.139353 6601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:02.126934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:54815 (local address 127.25.127.190:46035)
0619 08:26:02.126989 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:02.126991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:02.127003 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:26:02.127703 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:02.127705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:02.127720 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:02.127787 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:02.128655 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:02.129457 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:02.131098 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:02.131298 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:02.131353 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:02.133925 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:02.133926 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:02.133926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:02.133933 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:02.135864 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:02.136473 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:02.136475 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:02.136475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:02.136541 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:02.136859 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:02.136860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:02.136861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:02.137014 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:02.137082 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:02.139094 (+ 2012us) server_negotiation.cc:299] Negotiation successful
0619 08:26:02.139204 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:26:02.139473 6734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:02.126621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46035 (local address 127.25.127.130:54815)
0619 08:26:02.126845 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:02.126900 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:02.127586 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:02.127880 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:02.127885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:02.128130 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:02.128510 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:02.128518 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:02.129582 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:02.129582 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:02.130986 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:02.130990 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:02.131421 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:02.131423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:02.131577 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:02.132180 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:02.132192 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:02.133814 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:02.135960 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:02.135968 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:02.135971 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:02.136362 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:02.136616 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:02.136617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:02.136617 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:02.136763 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:02.138655 (+ 1892us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:02.138663 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:02.138886 (+ 223us) client_negotiation.cc:769] Sending connection context
0619 08:26:02.139019 (+ 133us) client_negotiation.cc:240] Negotiation successful
0619 08:26:02.139213 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":97,"threads_started":1}
I20260619 08:26:02.140228 6732 heartbeater.cc:347] Connected to a master server at 127.25.127.190:46035
I20260619 08:26:02.140343 6732 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:02.140558 6732 heartbeater.cc:511] Master 127.25.127.190:46035 requested a full tablet report, sending...
W20260619 08:26:02.141317 6732 heartbeater.cc:644] Failed to heartbeat to 127.25.127.190:46035 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 08:26:02.141396 6732 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:02.141465 6732 heartbeater.cc:511] Master 127.25.127.190:46035 requested a full tablet report, sending...
I20260619 08:26:02.141858 6732 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:02.141933 6732 heartbeater.cc:511] Master 127.25.127.190:46035 requested a full tablet report, sending...
W20260619 08:26:02.142282 6732 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 08:26:02.233595 6735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:02.233842 6735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:02.233883 6735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:02.235121 6735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:02.235195 6735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:02.235239 6735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:26:02.236827 6735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.127.148:44351/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:46299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=44855
--tserver_master_addrs=127.25.127.190:46035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:02.237133 6735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:02.237391 6735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:02.243219 6743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:02.243225 6741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:02.244189 6740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:02.245018 6735 server_base.cc:1034] running on GCE node
I20260619 08:26:02.245254 6735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:02.245563 6735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:02.246743 6735 hybrid_clock.cc:648] HybridClock initialized: now 1781857562246726 us; error 30 us; skew 500 ppm
Jun 19 08:26:02 dist-test-slave-5rhg krb5kdc[5395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857562, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:02.250445 6735 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:26:02.251020 6735 webserver.cc:460] Webserver started at http://127.25.127.131:44855/ using document root <none> and password file <none>
I20260619 08:26:02.251307 6735 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:02.251384 6735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 08:26:02 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857562, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
I20260619 08:26:02.269174 6735 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:26:02.271616 6752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:02.275149 6735 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.001s
I20260619 08:26:02.275276 6735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "d6cef17caf934179bfc9205a06befc1a"
format_stamp: "Formatted at 2026-06-19 08:26:01 on dist-test-slave-5rhg"
server_key: "cb795744aead814ed7378ad37f83110d"
server_key_iv: "8a2706b180b21383ef86a8c085cd3e07"
server_key_version: "kuduclusterkey@0"
I20260619 08:26:02.275774 6735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:02.287698 6735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:02.290153 6735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:02.290282 6735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:02.290568 6735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:02.290961 6735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:26:02.290997 6735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:02.291021 6735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:26:02.291034 6735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:02.298372 6735 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:46299
I20260619 08:26:02.298448 6866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:46299 every 8 connection(s)
I20260619 08:26:02.298825 6735 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:26:02.300549 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 6735
Jun 19 08:26:02 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857562, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:02.309799 6601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:02.299607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:51437 (local address 127.25.127.190:46035)
0619 08:26:02.299652 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:02.299654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:02.299881 (+ 227us) server_negotiation.cc:407] Connection header received
0619 08:26:02.300619 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:02.300621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:02.300635 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:02.300702 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:02.301478 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:02.301967 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:02.303408 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:02.303730 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:02.303798 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:02.306267 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:02.306269 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:02.306269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:02.306276 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:02.308090 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:02.308673 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:02.308675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:02.308675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:02.308736 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:02.309038 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:02.309039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:02.309040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:02.309173 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:02.309228 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:02.309634 (+ 406us) server_negotiation.cc:299] Negotiation successful
0619 08:26:02.309721 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:26:02.310045 6869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:02.299594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46035 (local address 127.25.127.131:51437)
0619 08:26:02.299755 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:02.299796 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:02.300452 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:02.300778 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:02.300782 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:02.301042 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:02.301363 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:02.301372 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:02.302071 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:02.302072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:02.303313 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:02.303316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:02.303843 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:02.303844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:02.303974 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:02.304546 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:02.304563 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:02.306156 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:02.308226 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:02.308235 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:02.308238 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:02.308575 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:02.308815 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:02.308816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:02.308816 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:02.308960 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:02.309264 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:02.309270 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:02.309448 (+ 178us) client_negotiation.cc:769] Sending connection context
0619 08:26:02.309601 (+ 153us) client_negotiation.cc:240] Negotiation successful
0619 08:26:02.309778 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260619 08:26:02.310608 6867 heartbeater.cc:347] Connected to a master server at 127.25.127.190:46035
I20260619 08:26:02.310719 6867 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:02.310918 6867 heartbeater.cc:511] Master 127.25.127.190:46035 requested a full tablet report, sending...
W20260619 08:26:02.311638 6867 heartbeater.cc:644] Failed to heartbeat to 127.25.127.190:46035 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 08:26:02.311710 6867 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:02.311771 6867 heartbeater.cc:511] Master 127.25.127.190:46035 requested a full tablet report, sending...
I20260619 08:26:02.312222 6867 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:02.312295 6867 heartbeater.cc:511] Master 127.25.127.190:46035 requested a full tablet report, sending...
W20260619 08:26:02.312683 6867 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 08:26:02.314283 6870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:02.300979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55696 (local address 127.25.127.190:46035)
0619 08:26:02.301309 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:02.301311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:02.301318 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:26:02.301339 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:02.301340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:02.301351 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:02.301420 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:02.303150 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:02.303737 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:02.304848 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:02.305173 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:02.305257 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:02.307325 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:02.307326 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:02.307327 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:02.307333 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:02.309044 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:02.309408 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:02.309409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:02.309410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:02.309467 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:02.309954 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:02.309956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:02.309956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:02.310080 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:02.310298 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:02.314059 (+ 3761us) server_negotiation.cc:299] Negotiation successful
0619 08:26:02.314159 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":68,"threads_started":1}
Jun 19 08:26:02 dist-test-slave-5rhg krb5kdc[5395](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781857562, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:02.972704 6597 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:02.974071 6597 heartbeater.cc:511] Master 127.25.127.190:46035 requested a full tablet report, sending...
I20260619 08:26:02.974581 6396 ts_manager.cc:194] Registered new tserver with Master: 1cab943d9fe04d65bb1772bba20458f4 (127.25.127.129:33971)
I20260619 08:26:02.975629 6396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:51781
I20260619 08:26:03.142482 6732 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:03.142658 6732 heartbeater.cc:511] Master 127.25.127.190:46035 requested a full tablet report, sending...
I20260619 08:26:03.143049 6396 ts_manager.cc:194] Registered new tserver with Master: 0b58ced57a8244b791103f7e3b45a7f7 (127.25.127.130:45149)
I20260619 08:26:03.143642 6396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:54815
Jun 19 08:26:03 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857562, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
I20260619 08:26:03.312865 6867 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:03.313060 6867 heartbeater.cc:511] Master 127.25.127.190:46035 requested a full tablet report, sending...
I20260619 08:26:03.313454 6396 ts_manager.cc:194] Registered new tserver with Master: d6cef17caf934179bfc9205a06befc1a (127.25.127.131:46299)
I20260619 08:26:03.313869 6396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:51437
I20260619 08:26:03.317476 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:26:03.326391 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:03.318810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55698 (local address 127.25.127.190:46035)
0619 08:26:03.318952 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:03.318954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:03.319034 (+ 80us) server_negotiation.cc:407] Connection header received
0619 08:26:03.319126 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:03.319127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:03.319140 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:03.319212 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:03.319947 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.320563 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.321355 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.321586 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.321682 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:03.323117 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:03.323120 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:03.323120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:03.323129 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:03.324988 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:03.325411 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:03.325413 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:03.325413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:03.325496 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:03.325756 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:03.325757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:03.325757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:03.325909 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:03.326032 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:03.326153 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 08:26:03.326256 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
I20260619 08:26:03.328807 6396 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:26:03.331106 6396 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:26:03.340674 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:03.334255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:33971 (local address 127.0.0.1:59150)
0619 08:26:03.336264 (+ 2009us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:03.336273 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:03.336315 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:03.336568 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:03.336572 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:03.336575 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:03.336664 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:03.336666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.337958 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.337959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:03.339258 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:03.339261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.340352 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.340353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:03.340442 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:03.340458 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 08:26:03.340528 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 08:26:03.340580 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1986,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260619 08:26:03.341303 6896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:03.334536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59150 (local address 127.25.127.129:33971)
0619 08:26:03.336405 (+ 1869us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:03.336409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:03.336418 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:26:03.336449 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:03.336450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:03.336463 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:03.336558 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:03.336759 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.337866 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.339349 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.340182 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.340250 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:03.340319 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:03.341170 (+ 851us) server_negotiation.cc:299] Negotiation successful
0619 08:26:03.341218 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1836,"thread_start_us":65,"threads_started":1}
I20260619 08:26:03.341482 6899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:03.334876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59210 (local address 127.25.127.131:46299)
0619 08:26:03.335225 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:03.335230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:03.336797 (+ 1567us) server_negotiation.cc:407] Connection header received
0619 08:26:03.337203 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:03.337206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:03.337224 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:03.337282 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:03.337515 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.338581 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.340246 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.341030 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.341086 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:03.341142 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:03.341354 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 08:26:03.341402 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":56,"threads_started":1}
I20260619 08:26:03.341694 6894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:03.334780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:46299 (local address 127.0.0.1:59210)
0619 08:26:03.336733 (+ 1953us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:03.336743 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:03.336825 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:03.337397 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:03.337398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:03.337400 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:03.337441 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:03.337442 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.338679 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.338679 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:03.340092 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:03.340095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.341204 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.341205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:03.341264 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:03.341268 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:26:03.341549 (+ 281us) client_negotiation.cc:240] Negotiation successful
0619 08:26:03.341596 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1935,"thread_start_us":73,"threads_started":1}
I20260619 08:26:03.342486 6800 tablet_service.cc:1467] Processing CreateTablet for tablet 89cc56401a2e426b997c30082b4d51ae (DEFAULT_TABLE table=test-table [id=d4cc7add51e342cd8bbfa08273d23330]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:26:03.342743 6529 tablet_service.cc:1467] Processing CreateTablet for tablet 89cc56401a2e426b997c30082b4d51ae (DEFAULT_TABLE table=test-table [id=d4cc7add51e342cd8bbfa08273d23330]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:26:03.342901 6800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89cc56401a2e426b997c30082b4d51ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:03.343082 6529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89cc56401a2e426b997c30082b4d51ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:03.346444 6898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:03.334255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:45149 (local address 127.0.0.1:54630)
0619 08:26:03.334969 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:03.335018 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:03.335088 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:03.342405 (+ 7317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:03.342405 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:03.342408 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:03.342449 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:03.342451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.343736 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.343737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:03.344823 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:03.344826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.346200 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.346201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:03.346270 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:03.346276 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 08:26:03.346312 (+ 36us) client_negotiation.cc:240] Negotiation successful
0619 08:26:03.346357 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":87,"threads_started":1}
I20260619 08:26:03.346822 6897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:03.334549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54630 (local address 127.25.127.130:45149)
0619 08:26:03.341918 (+ 7369us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:03.341924 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:03.341934 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:26:03.341970 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:03.341972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:03.342302 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:03.342561 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:03.342654 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.343628 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.344951 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.346097 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.346572 (+ 475us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:03.346642 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:03.346672 (+ 30us) server_negotiation.cc:299] Negotiation successful
0619 08:26:03.346718 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7320,"thread_start_us":56,"threads_started":1}
I20260619 08:26:03.347497 6650 tablet_service.cc:1467] Processing CreateTablet for tablet 89cc56401a2e426b997c30082b4d51ae (DEFAULT_TABLE table=test-table [id=d4cc7add51e342cd8bbfa08273d23330]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:26:03.347862 6650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89cc56401a2e426b997c30082b4d51ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:03.350909 6902 tablet_bootstrap.cc:492] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7: Bootstrap starting.
I20260619 08:26:03.351600 6902 tablet_bootstrap.cc:654] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:03.351909 6902 log.cc:824] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:03.352218 6900 tablet_bootstrap.cc:492] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a: Bootstrap starting.
I20260619 08:26:03.352690 6902 tablet_bootstrap.cc:492] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7: No bootstrap required, opened a new log
I20260619 08:26:03.352782 6902 ts_tablet_manager.cc:1397] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:26:03.352903 6900 tablet_bootstrap.cc:654] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:03.353224 6900 log.cc:824] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:03.355077 6902 raft_consensus.cc:348] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b58ced57a8244b791103f7e3b45a7f7" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45149 } } peers { permanent_uuid: "1cab943d9fe04d65bb1772bba20458f4" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33971 } } peers { permanent_uuid: "d6cef17caf934179bfc9205a06befc1a" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46299 } }
I20260619 08:26:03.355299 6902 raft_consensus.cc:374] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:03.355381 6902 raft_consensus.cc:729] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b58ced57a8244b791103f7e3b45a7f7, State: Initialized, Role: FOLLOWER
I20260619 08:26:03.355527 6902 consensus_queue.cc:260] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b58ced57a8244b791103f7e3b45a7f7" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45149 } } peers { permanent_uuid: "1cab943d9fe04d65bb1772bba20458f4" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33971 } } peers { permanent_uuid: "d6cef17caf934179bfc9205a06befc1a" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46299 } }
I20260619 08:26:03.355860 6902 ts_tablet_manager.cc:1428] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:26:03.355865 6732 heartbeater.cc:503] Master 127.25.127.190:46035 was elected leader, sending a full tablet report...
I20260619 08:26:03.360646 6900 tablet_bootstrap.cc:492] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a: No bootstrap required, opened a new log
I20260619 08:26:03.360762 6900 ts_tablet_manager.cc:1397] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20260619 08:26:03.362972 6900 raft_consensus.cc:348] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b58ced57a8244b791103f7e3b45a7f7" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45149 } } peers { permanent_uuid: "1cab943d9fe04d65bb1772bba20458f4" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33971 } } peers { permanent_uuid: "d6cef17caf934179bfc9205a06befc1a" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46299 } }
I20260619 08:26:03.363168 6900 raft_consensus.cc:374] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:03.363216 6900 raft_consensus.cc:729] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6cef17caf934179bfc9205a06befc1a, State: Initialized, Role: FOLLOWER
I20260619 08:26:03.363371 6900 consensus_queue.cc:260] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b58ced57a8244b791103f7e3b45a7f7" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45149 } } peers { permanent_uuid: "1cab943d9fe04d65bb1772bba20458f4" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33971 } } peers { permanent_uuid: "d6cef17caf934179bfc9205a06befc1a" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46299 } }
I20260619 08:26:03.363691 6900 ts_tablet_manager.cc:1428] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 08:26:03.363838 6867 heartbeater.cc:503] Master 127.25.127.190:46035 was elected leader, sending a full tablet report...
I20260619 08:26:03.363996 6901 tablet_bootstrap.cc:492] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4: Bootstrap starting.
I20260619 08:26:03.364801 6901 tablet_bootstrap.cc:654] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:03.365129 6901 log.cc:824] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:03.369267 6901 tablet_bootstrap.cc:492] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4: No bootstrap required, opened a new log
I20260619 08:26:03.369367 6901 ts_tablet_manager.cc:1397] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 08:26:03.371553 6901 raft_consensus.cc:348] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b58ced57a8244b791103f7e3b45a7f7" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45149 } } peers { permanent_uuid: "1cab943d9fe04d65bb1772bba20458f4" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33971 } } peers { permanent_uuid: "d6cef17caf934179bfc9205a06befc1a" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46299 } }
I20260619 08:26:03.371757 6901 raft_consensus.cc:374] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:03.371834 6901 raft_consensus.cc:729] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cab943d9fe04d65bb1772bba20458f4, State: Initialized, Role: FOLLOWER
I20260619 08:26:03.372018 6901 consensus_queue.cc:260] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b58ced57a8244b791103f7e3b45a7f7" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45149 } } peers { permanent_uuid: "1cab943d9fe04d65bb1772bba20458f4" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33971 } } peers { permanent_uuid: "d6cef17caf934179bfc9205a06befc1a" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46299 } }
I20260619 08:26:03.372347 6901 ts_tablet_manager.cc:1428] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:26:03.372490 6597 heartbeater.cc:503] Master 127.25.127.190:46035 was elected leader, sending a full tablet report...
W20260619 08:26:03.379244 6733 tablet.cc:2369] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:26:03.450096 6598 tablet.cc:2369] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 08:26:03 dist-test-slave-5rhg krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857562, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for HTTP/127.25.127.148@KRBTEST.COM
W20260619 08:26:03.550197 6868 tablet.cc:2369] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:26:03.640560 6908 raft_consensus.cc:482] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:26:03.640728 6908 raft_consensus.cc:504] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b58ced57a8244b791103f7e3b45a7f7" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45149 } } peers { permanent_uuid: "1cab943d9fe04d65bb1772bba20458f4" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33971 } } peers { permanent_uuid: "d6cef17caf934179bfc9205a06befc1a" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46299 } }
I20260619 08:26:03.641170 6908 leader_election.cc:288] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b58ced57a8244b791103f7e3b45a7f7 (127.25.127.130:45149), d6cef17caf934179bfc9205a06befc1a (127.25.127.131:46299)
I20260619 08:26:03.648346 6897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:03.641719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:43433 (local address 127.25.127.130:45149)
0619 08:26:03.641767 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:03.641769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:03.644605 (+ 2836us) server_negotiation.cc:407] Connection header received
0619 08:26:03.644709 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:03.644711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:03.644720 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:03.644752 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:03.644994 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.645723 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.647250 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.647815 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.647871 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:03.647882 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:03.648137 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 08:26:03.648216 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:26:03.648875 6911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:03.641516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:45149 (local address 127.25.127.129:43433)
0619 08:26:03.644549 (+ 3033us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:03.644559 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:03.644631 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:03.644821 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:03.644822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:03.644825 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:03.644907 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:03.644910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.645841 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.645842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:03.647132 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:03.647135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.647971 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.647972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:03.648043 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:03.648047 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:26:03.648716 (+ 669us) client_negotiation.cc:240] Negotiation successful
0619 08:26:03.648770 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3010,"thread_start_us":82,"threads_started":1}
I20260619 08:26:03.649487 6685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89cc56401a2e426b997c30082b4d51ae" candidate_uuid: "1cab943d9fe04d65bb1772bba20458f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b58ced57a8244b791103f7e3b45a7f7" is_pre_election: true
I20260619 08:26:03.649660 6685 raft_consensus.cc:2454] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cab943d9fe04d65bb1772bba20458f4 in term 0.
I20260619 08:26:03.649992 6483 leader_election.cc:302] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b58ced57a8244b791103f7e3b45a7f7, 1cab943d9fe04d65bb1772bba20458f4; no voters:
I20260619 08:26:03.650127 6908 raft_consensus.cc:2781] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:26:03.650169 6908 raft_consensus.cc:482] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:26:03.650192 6908 raft_consensus.cc:3037] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:03.651100 6908 raft_consensus.cc:504] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b58ced57a8244b791103f7e3b45a7f7" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45149 } } peers { permanent_uuid: "1cab943d9fe04d65bb1772bba20458f4" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33971 } } peers { permanent_uuid: "d6cef17caf934179bfc9205a06befc1a" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46299 } }
I20260619 08:26:03.651242 6908 leader_election.cc:288] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [CANDIDATE]: Term 1 election: Requested vote from peers 0b58ced57a8244b791103f7e3b45a7f7 (127.25.127.130:45149), d6cef17caf934179bfc9205a06befc1a (127.25.127.131:46299)
I20260619 08:26:03.651577 6685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89cc56401a2e426b997c30082b4d51ae" candidate_uuid: "1cab943d9fe04d65bb1772bba20458f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b58ced57a8244b791103f7e3b45a7f7"
I20260619 08:26:03.651661 6685 raft_consensus.cc:3037] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:03.652082 6899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:03.641629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:60645 (local address 127.25.127.131:46299)
0619 08:26:03.641676 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:03.641679 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:03.644413 (+ 2734us) server_negotiation.cc:407] Connection header received
0619 08:26:03.644505 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:03.644507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:03.644520 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:03.644578 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:03.649135 (+ 4557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.649792 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.651118 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.651668 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.651716 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:03.651725 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:03.651972 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 08:26:03.652017 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:26:03.652341 6912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:03.641359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:46299 (local address 127.25.127.129:60645)
0619 08:26:03.644308 (+ 2949us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:03.644323 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:03.644434 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:03.648962 (+ 4528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:03.648964 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:03.648967 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:03.649030 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:03.649031 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.649917 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.649918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:03.651001 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:03.651004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.651787 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.651788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:03.651869 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:03.651886 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 08:26:03.652210 (+ 324us) client_negotiation.cc:240] Negotiation successful
0619 08:26:03.652264 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2924,"thread_start_us":103,"threads_started":1}
I20260619 08:26:03.652896 6685 raft_consensus.cc:2454] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cab943d9fe04d65bb1772bba20458f4 in term 1.
I20260619 08:26:03.652916 6821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89cc56401a2e426b997c30082b4d51ae" candidate_uuid: "1cab943d9fe04d65bb1772bba20458f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6cef17caf934179bfc9205a06befc1a" is_pre_election: true
I20260619 08:26:03.653105 6821 raft_consensus.cc:2454] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cab943d9fe04d65bb1772bba20458f4 in term 0.
I20260619 08:26:03.653143 6483 leader_election.cc:302] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b58ced57a8244b791103f7e3b45a7f7, 1cab943d9fe04d65bb1772bba20458f4; no voters:
I20260619 08:26:03.653225 6908 raft_consensus.cc:2781] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:26:03.653254 6819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89cc56401a2e426b997c30082b4d51ae" candidate_uuid: "1cab943d9fe04d65bb1772bba20458f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6cef17caf934179bfc9205a06befc1a"
I20260619 08:26:03.653322 6819 raft_consensus.cc:3037] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:03.653400 6908 raft_consensus.cc:686] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [term 1 LEADER]: Becoming Leader. State: Replica: 1cab943d9fe04d65bb1772bba20458f4, State: Running, Role: LEADER
I20260619 08:26:03.653505 6908 consensus_queue.cc:237] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b58ced57a8244b791103f7e3b45a7f7" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45149 } } peers { permanent_uuid: "1cab943d9fe04d65bb1772bba20458f4" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33971 } } peers { permanent_uuid: "d6cef17caf934179bfc9205a06befc1a" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46299 } }
I20260619 08:26:03.654209 6819 raft_consensus.cc:2454] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cab943d9fe04d65bb1772bba20458f4 in term 1.
I20260619 08:26:03.654397 6396 catalog_manager.cc:5495] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cab943d9fe04d65bb1772bba20458f4 (127.25.127.129). New cstate: current_term: 1 leader_uuid: "1cab943d9fe04d65bb1772bba20458f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b58ced57a8244b791103f7e3b45a7f7" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cab943d9fe04d65bb1772bba20458f4" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6cef17caf934179bfc9205a06befc1a" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46299 } health_report { overall_health: UNKNOWN } } }
I20260619 08:26:03.693238 6896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:03.689607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59164 (local address 127.25.127.129:33971)
0619 08:26:03.689650 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:03.689652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:03.689660 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:26:03.689703 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:03.689704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:03.689727 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:03.689795 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:26:03.689982 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.690749 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.691898 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:03.692232 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:03.692314 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:03.692553 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:26:03.692731 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:26:03.692831 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:03.693113 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 08:26:03.693138 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:26:03.695845 6819 raft_consensus.cc:1261] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a [term 1 FOLLOWER]: Refusing update from remote peer 1cab943d9fe04d65bb1772bba20458f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:26:03.696256 6685 raft_consensus.cc:1261] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7 [term 1 FOLLOWER]: Refusing update from remote peer 1cab943d9fe04d65bb1772bba20458f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:26:03.696812 6913 consensus_queue.cc:1035] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6cef17caf934179bfc9205a06befc1a" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:26:03.697347 6913 consensus_queue.cc:1035] T 89cc56401a2e426b997c30082b4d51ae P 1cab943d9fe04d65bb1772bba20458f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b58ced57a8244b791103f7e3b45a7f7" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:26:03.704644 6916 mvcc.cc:205] Tried to move back new op lower bound from 7298488580894486528 to 7298488580725329920. Current Snapshot: MvccSnapshot[applied={T|T < 7298488580894486528}]
I20260619 08:26:03.704864 6919 mvcc.cc:205] Tried to move back new op lower bound from 7298488580894486528 to 7298488580725329920. Current Snapshot: MvccSnapshot[applied={T|T < 7298488580894486528}]
I20260619 08:26:03.707804 6917 mvcc.cc:205] Tried to move back new op lower bound from 7298488580894486528 to 7298488580725329920. Current Snapshot: MvccSnapshot[applied={T|T < 7298488580894486528}]
I20260619 08:26:03.708580 6396 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:26:03.708684 6396 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:26:03.709578 6396 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 943dfa643151426eae072de881a05f2d: Sending DeleteTablet for 3 replicas of tablet 89cc56401a2e426b997c30082b4d51ae
I20260619 08:26:03.710055 6650 tablet_service.cc:1514] Processing DeleteTablet for tablet 89cc56401a2e426b997c30082b4d51ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:26:03 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:54630
I20260619 08:26:03.710307 6529 tablet_service.cc:1514] Processing DeleteTablet for tablet 89cc56401a2e426b997c30082b4d51ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:26:03 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:59150
I20260619 08:26:03.710836 6800 tablet_service.cc:1514] Processing DeleteTablet for tablet 89cc56401a2e426b997c30082b4d51ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:26:03 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:59210
I20260619 08:26:03.711525 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 6435
I20260619 08:26:03.716313 6927 tablet_replica.cc:330] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a: stopping tablet replica
I20260619 08:26:03.716529 6927 raft_consensus.cc:2229] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:26:03.716799 6927 raft_consensus.cc:2258] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:26:03.717094 6925 tablet_replica.cc:330] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7: stopping tablet replica
I20260619 08:26:03.717593 6927 ts_tablet_manager.cc:1905] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:26:03.723080 6925 raft_consensus.cc:2229] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:26:03.723381 6925 raft_consensus.cc:2258] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:26:03.724215 6927 ts_tablet_manager.cc:1918] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:26:03.724778 6925 ts_tablet_manager.cc:1905] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:26:03.726445 6925 ts_tablet_manager.cc:1918] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:26:03.726644 6927 log.cc:1196] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/89cc56401a2e426b997c30082b4d51ae
I20260619 08:26:03.727010 6927 ts_tablet_manager.cc:1939] T 89cc56401a2e426b997c30082b4d51ae P d6cef17caf934179bfc9205a06befc1a: Deleting consensus metadata
I20260619 08:26:03.727622 6382 catalog_manager.cc:4841] TS d6cef17caf934179bfc9205a06befc1a (127.25.127.131:46299): tablet 89cc56401a2e426b997c30082b4d51ae (table test-table [id=d4cc7add51e342cd8bbfa08273d23330]) successfully deleted
I20260619 08:26:03.727834 6925 log.cc:1196] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/89cc56401a2e426b997c30082b4d51ae
I20260619 08:26:03.728222 6925 ts_tablet_manager.cc:1939] T 89cc56401a2e426b997c30082b4d51ae P 0b58ced57a8244b791103f7e3b45a7f7: Deleting consensus metadata
I20260619 08:26:03.728747 6381 catalog_manager.cc:4841] TS 0b58ced57a8244b791103f7e3b45a7f7 (127.25.127.130:45149): tablet 89cc56401a2e426b997c30082b4d51ae (table test-table [id=d4cc7add51e342cd8bbfa08273d23330]) successfully deleted
W20260619 08:26:03.729264 6380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.127.129:33971 (error 108)
I20260619 08:26:03.730209 6898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:03.730085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:33971 (local address 127.0.0.1:59178)
0619 08:26:03.730134 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:03.730172 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.129:33971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 08:26:03.730248 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 6599
W20260619 08:26:03.730938 6380 catalog_manager.cc:4584] TS 1cab943d9fe04d65bb1772bba20458f4 (127.25.127.129:33971): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 89cc56401a2e426b997c30082b4d51ae: Network error: Client connection negotiation failed: client connection to 127.25.127.129:33971: connect: Connection refused (error 111)
I20260619 08:26:03.737226 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 6735
I20260619 08:26:03.742983 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 6363
I20260619 08:26:03.750380 26110 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 08:26:04.095407 26110 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 08:26:04.095510 26110 mini_ranger.cc:66] Stopping Ranger...
I20260619 08:26:04.440897 26110 mini_ranger.cc:68] Stopped Ranger
2026-06-19 08:26:04.441 UTC [5425] LOG: received smart shutdown request
2026-06-19 08:26:04.444 UTC [5425] LOG: background worker "logical replication launcher" (PID 5434) exited with exit code 1
2026-06-19 08:26:04.445 UTC [5429] LOG: shutting down
2026-06-19 08:26:04.629 UTC [5425] LOG: database system is shut down
2026-06-19T08:26:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48413 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:26:04 dist-test-slave-5rhg krb5kdc[6937](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:26:04 dist-test-slave-5rhg krb5kdc[6937](info): set up 2 sockets
Jun 19 08:26:04 dist-test-slave-5rhg krb5kdc[6937](info): commencing operation
krb5kdc: starting...
W20260619 08:26:05.788988 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:26:05 dist-test-slave-5rhg krb5kdc[6937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:26:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:26:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:26:05.906917 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:33187
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:33855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:33187,127.25.127.189:44381,127.25.127.188:40137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:26:05.990767 6950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:05.991031 6950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:05.991073 6950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:05.992326 6950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:26:05.992393 6950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:05.992417 6950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:05.992436 6950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:26:05.992468 6950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:26:05.993963 6950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:33855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:33187,127.25.127.189:44381,127.25.127.188:40137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:33187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:05.994242 6950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:05.994479 6950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:05.999408 6957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:05.999408 6956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:05.999473 6959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:05.999471 6950 server_base.cc:1034] running on GCE node
I20260619 08:26:05.999958 6950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:06.000336 6950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:06.001499 6950 hybrid_clock.cc:648] HybridClock initialized: now 1781857566001480 us; error 36 us; skew 500 ppm
Jun 19 08:26:05 dist-test-slave-5rhg krb5kdc[6937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857565, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:06.003955 6950 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:26:06.004472 6950 webserver.cc:460] Webserver started at http://127.25.127.190:40415/ using document root <none> and password file <none>
I20260619 08:26:06.004721 6950 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:06.004792 6950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:06.004943 6950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:26:06.006042 6950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "f810b83bb28c4f9082ad2ee9f10e8a90"
format_stamp: "Formatted at 2026-06-19 08:26:06 on dist-test-slave-5rhg"
server_key: "82f371ec94049355bb773b855fa67c97"
server_key_iv: "160c18ce23de10bb953a4c027783c494"
server_key_version: "encryptionkey@0"
I20260619 08:26:06.006408 6950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "f810b83bb28c4f9082ad2ee9f10e8a90"
format_stamp: "Formatted at 2026-06-19 08:26:06 on dist-test-slave-5rhg"
server_key: "82f371ec94049355bb773b855fa67c97"
server_key_iv: "160c18ce23de10bb953a4c027783c494"
server_key_version: "encryptionkey@0"
I20260619 08:26:06.007725 6950 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:26:06.008493 6966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:06.008664 6950 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:06.008775 6950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "f810b83bb28c4f9082ad2ee9f10e8a90"
format_stamp: "Formatted at 2026-06-19 08:26:06 on dist-test-slave-5rhg"
server_key: "82f371ec94049355bb773b855fa67c97"
server_key_iv: "160c18ce23de10bb953a4c027783c494"
server_key_version: "encryptionkey@0"
I20260619 08:26:06.008853 6950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:06.023202 6950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:06.025400 6950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:06.025566 6950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:06.029842 6950 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:33187
I20260619 08:26:06.029878 7018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:33187 every 8 connection(s)
I20260619 08:26:06.030272 6950 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:26:06.031178 7019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:06.032238 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 6950
I20260619 08:26:06.032378 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:26:06.032562 26110 external_mini_cluster.cc:1428] Setting key a8d95bc6be2eb97f915d11af758c56bd
I20260619 08:26:06.032974 7019 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:26:06 dist-test-slave-5rhg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857565, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.127.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.189@KRBTEST.COM" created.
I20260619 08:26:06.044898 7022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.033740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54252 (local address 127.25.127.190:33187)
0619 08:26:06.033978 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:06.033990 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:06.034198 (+ 208us) server_negotiation.cc:407] Connection header received
0619 08:26:06.034709 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:06.034712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:06.034741 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:06.034856 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:06.035486 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.036469 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.038081 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.038406 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.038493 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.041086 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:06.041096 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:06.041096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:06.041106 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:06.042740 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.043275 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.043279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.043279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.043392 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.043721 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.043722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.043722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.044130 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:06.044244 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:06.044512 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 08:26:06.044678 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":196,"thread_start_us":86,"threads_started":1}
I20260619 08:26:06.045003 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.033722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:33187 (local address 127.0.0.1:54252)
0619 08:26:06.034060 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.034091 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:06.034548 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:06.034894 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:06.034896 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:06.035187 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:06.035365 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:06.035370 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.036571 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.036573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:06.037975 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:06.037978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.038506 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.038507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:06.038611 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.039201 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:06.039213 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:06.040987 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:06.042863 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:06.042870 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:06.042872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:06.043180 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:06.043485 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:06.043486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:06.043487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:06.043646 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:06.044239 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:06.044262 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:06.044350 (+ 88us) client_negotiation.cc:769] Sending connection context
0619 08:26:06.044466 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 08:26:06.044773 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":94,"thread_start_us":95,"threads_started":1}
I20260619 08:26:06.045784 7019 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:26:06.046146 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.046026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:44381 (local address 127.0.0.1:53842)
0619 08:26:06.046076 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.046122 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.189:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 08:26:06.046348 6968 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.127.189:44381: connect: Connection refused (error 111)
I20260619 08:26:06.046847 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.046729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:44381 (local address 127.0.0.1:53850)
0619 08:26:06.046783 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.046818 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.189:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 08:26:06.046983 7019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.189:44381: Network error: Client connection negotiation failed: client connection to 127.25.127.189:44381: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.189.keytab.
Entry for principal kudu/127.25.127.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.189.keytab.
I20260619 08:26:06.061436 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.189:44381
--webserver_interface=127.25.127.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.189.keytab
--principal=kudu/127.25.127.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:33855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:33187,127.25.127.189:44381,127.25.127.188:40137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:26:06.096256 7019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } attempt: 1
I20260619 08:26:06.096808 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.096643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:44381 (local address 127.0.0.1:53858)
0619 08:26:06.096713 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.096764 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.189:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 08:26:06.097199 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.097110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:44381 (local address 127.0.0.1:53866)
0619 08:26:06.097153 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.097180 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.189:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 08:26:06.097313 7019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.189:44381: Network error: Client connection negotiation failed: client connection to 127.25.127.189:44381: connect: Connection refused (error 111)
W20260619 08:26:06.147621 7026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:06.147840 7026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:06.147866 7026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:06.149163 7026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:26:06.149207 7026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:06.149222 7026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:06.149272 7026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:26:06.149291 7026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:26:06.150796 7026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:33855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:33187,127.25.127.189:44381,127.25.127.188:40137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.189.keytab
--principal=kudu/127.25.127.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.189:44381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.127.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:06.151085 7026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:06.151332 7026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:06.156268 7032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:06.156289 7031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:06.156318 7026 server_base.cc:1034] running on GCE node
W20260619 08:26:06.156286 7034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:06.156828 7026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:06.157217 7026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:06.158386 7026 hybrid_clock.cc:648] HybridClock initialized: now 1781857566158374 us; error 47 us; skew 500 ppm
Jun 19 08:26:06 dist-test-slave-5rhg krb5kdc[6937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857566, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:06.161038 7026 init.cc:377] Logged in from keytab as kudu/127.25.127.189@KRBTEST.COM (short username kudu)
I20260619 08:26:06.161501 7026 webserver.cc:460] Webserver started at http://127.25.127.189:42221/ using document root <none> and password file <none>
I20260619 08:26:06.161743 7026 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:06.161813 7026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:06.161973 7026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:26:06.163039 7026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-1/data/instance:
uuid: "5dc79d4f00da44bf878c14c5608db92d"
format_stamp: "Formatted at 2026-06-19 08:26:06 on dist-test-slave-5rhg"
server_key: "d9181c2b061002182af80656f42c5b40"
server_key_iv: "361f490985eab713fcfe0158e24c5d2a"
server_key_version: "encryptionkey@0"
I20260619 08:26:06.163403 7026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-1/wal/instance:
uuid: "5dc79d4f00da44bf878c14c5608db92d"
format_stamp: "Formatted at 2026-06-19 08:26:06 on dist-test-slave-5rhg"
server_key: "d9181c2b061002182af80656f42c5b40"
server_key_iv: "361f490985eab713fcfe0158e24c5d2a"
server_key_version: "encryptionkey@0"
I20260619 08:26:06.164825 7026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:26:06.165498 7019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } attempt: 2
I20260619 08:26:06.165803 7041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:06.165871 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.165746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:44381 (local address 127.0.0.1:53874)
0619 08:26:06.165806 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.165835 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.189:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 08:26:06.166021 7026 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:06.166116 7026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-1/wal
uuid: "5dc79d4f00da44bf878c14c5608db92d"
format_stamp: "Formatted at 2026-06-19 08:26:06 on dist-test-slave-5rhg"
server_key: "d9181c2b061002182af80656f42c5b40"
server_key_iv: "361f490985eab713fcfe0158e24c5d2a"
server_key_version: "encryptionkey@0"
I20260619 08:26:06.166242 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.166135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:44381 (local address 127.0.0.1:53882)
0619 08:26:06.166166 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.166214 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.189:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260619 08:26:06.166203 7026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260619 08:26:06.166337 7019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.189:44381: Network error: Client connection negotiation failed: client connection to 127.25.127.189:44381: connect: Connection refused (error 111)
I20260619 08:26:06.178243 7026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:06.180310 7026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:06.180433 7026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:06.184358 7026 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.189:44381
I20260619 08:26:06.184392 7093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.189:44381 every 8 connection(s)
I20260619 08:26:06.184778 7026 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-1/data/info.pb
I20260619 08:26:06.185675 7094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:06.186964 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 7026
I20260619 08:26:06.187095 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-1/wal/instance
I20260619 08:26:06.187273 26110 external_mini_cluster.cc:1428] Setting key f33236012c3a283200d22c7cde06716a
I20260619 08:26:06.187394 7094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:26:06 dist-test-slave-5rhg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857566, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.189@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.127.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.188@KRBTEST.COM" created.
I20260619 08:26:06.197054 7022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.188208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54260 (local address 127.25.127.190:33187)
0619 08:26:06.188280 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:06.188283 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:06.188498 (+ 215us) server_negotiation.cc:407] Connection header received
0619 08:26:06.189227 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:06.189228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:06.189242 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:06.189296 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:06.190131 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.190640 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.191667 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.191897 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.191979 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.194149 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:06.194151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:06.194152 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:06.194159 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:06.195616 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.196037 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.196038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.196039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.196100 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.196387 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.196388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.196388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.196509 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:06.196579 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:06.196862 (+ 283us) server_negotiation.cc:299] Negotiation successful
0619 08:26:06.196958 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 08:26:06.197249 7096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.188143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:33187 (local address 127.0.0.1:54260)
0619 08:26:06.188345 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.188382 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:06.189099 (+ 717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:06.189401 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:06.189406 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:06.189782 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:06.190032 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:06.190037 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.190741 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.190742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:06.191572 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:06.191574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.191985 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.191987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:06.192129 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.192722 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:06.192730 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:06.194050 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:06.195718 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:06.195726 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:06.195728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:06.195959 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:06.196190 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:06.196190 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:06.196191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:06.196310 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:06.196582 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:06.196589 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:06.196733 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 08:26:06.196843 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 08:26:06.197007 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
I20260619 08:26:06.197837 7094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:26:06 dist-test-slave-5rhg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857566, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.189@KRBTEST.COM for kudu/127.25.127.189@KRBTEST.COM
I20260619 08:26:06.205616 7096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.198110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:44381 (local address 127.0.0.1:53886)
0619 08:26:06.198159 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.198170 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:06.198210 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:06.198497 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:06.198498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:06.198634 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:06.198705 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:06.198707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.199622 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.199623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:06.200203 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:06.200205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.200882 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.200883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:06.200942 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.201365 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:06.201369 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:06.202493 (+ 1124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:06.204496 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:06.204497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:06.204497 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:06.204650 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:06.204928 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:06.204929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:06.204929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:06.204991 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:06.205351 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:06.205351 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:06.205398 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 08:26:06.205458 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 08:26:06.205529 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 08:26:06.205739 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.198167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53886 (local address 127.25.127.189:44381)
0619 08:26:06.198338 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:06.198343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:06.198366 (+ 23us) server_negotiation.cc:407] Connection header received
0619 08:26:06.198389 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:06.198392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:06.198407 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:06.198487 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:06.198792 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.199510 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.200286 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.200788 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.200877 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.202575 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:06.202584 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:06.202584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:06.202592 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:06.204414 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.204764 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.204767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.204768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.204831 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.205066 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.205067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.205067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.205261 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:06.205356 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:06.205512 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 08:26:06.205574 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260619 08:26:06.206315 7094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:26:06.206700 7096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.206594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:40137 (local address 127.0.0.1:48616)
0619 08:26:06.206640 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.206683 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:40137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 08:26:06.206871 7044 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.127.188:40137: connect: Connection refused (error 111)
I20260619 08:26:06.207332 7096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.207253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:40137 (local address 127.0.0.1:48624)
0619 08:26:06.207294 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.207317 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:40137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 08:26:06.207445 7094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.188:40137: Network error: Client connection negotiation failed: client connection to 127.25.127.188:40137: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.188.keytab.
Entry for principal kudu/127.25.127.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.188.keytab.
I20260619 08:26:06.212960 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.188:40137
--webserver_interface=127.25.127.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.188.keytab
--principal=kudu/127.25.127.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:33855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:33187,127.25.127.189:44381,127.25.127.188:40137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:26:06.256706 7094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } attempt: 1
I20260619 08:26:06.257257 7096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.257122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:40137 (local address 127.0.0.1:48636)
0619 08:26:06.257185 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.257227 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:40137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 08:26:06.257493 7019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } attempt: 3
I20260619 08:26:06.257645 7096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.257538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:40137 (local address 127.0.0.1:48648)
0619 08:26:06.257586 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.257616 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:40137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 08:26:06.257736 7094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.188:40137: Network error: Client connection negotiation failed: client connection to 127.25.127.188:40137: connect: Connection refused (error 111)
Jun 19 08:26:06 dist-test-slave-5rhg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857565, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for kudu/127.25.127.189@KRBTEST.COM
I20260619 08:26:06.265033 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.257784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:44381 (local address 127.0.0.1:53888)
0619 08:26:06.257840 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.257866 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:06.257913 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:06.258240 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:06.258240 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:06.258456 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:06.258539 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:06.258543 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.259517 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.259518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:06.260040 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:06.260041 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.260560 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.260561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:06.260628 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.261127 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:06.261133 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:06.262348 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:06.264029 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:06.264030 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:06.264031 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:06.264282 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:06.264484 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:06.264485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:06.264485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:06.264538 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:06.264789 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:06.264790 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:06.264829 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 08:26:06.264879 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 08:26:06.264963 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 08:26:06.265102 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.258035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53888 (local address 127.25.127.189:44381)
0619 08:26:06.258108 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:06.258110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:06.258121 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:26:06.258147 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:06.258148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:06.258158 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:06.258220 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:06.258609 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.259411 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.260131 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.260483 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.260554 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.262434 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:06.262436 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:06.262436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:06.262444 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:06.263947 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.264352 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.264354 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.264354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.264418 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.264596 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.264597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.264598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.264731 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:06.264796 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:06.264925 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 08:26:06.265009 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 08:26:06.265468 7019 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:26:06.265795 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.265698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:40137 (local address 127.0.0.1:48664)
0619 08:26:06.265744 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.265775 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:40137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 08:26:06.266077 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.266024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:40137 (local address 127.0.0.1:48670)
0619 08:26:06.266045 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.266065 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:40137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260619 08:26:06.266182 7019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.188:40137: Network error: Client connection negotiation failed: client connection to 127.25.127.188:40137: connect: Connection refused (error 111)
W20260619 08:26:06.296363 7101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:06.296576 7101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:06.296597 7101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 08:26:06.297370 7019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } attempt: 1
I20260619 08:26:06.297808 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.297666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:40137 (local address 127.0.0.1:48676)
0619 08:26:06.297732 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.297776 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:40137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 08:26:06.297879 7101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:26:06.297957 7101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:06.297982 7101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:06.298012 7101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:26:06.298040 7101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:26:06.298195 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.298106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:40137 (local address 127.0.0.1:48690)
0619 08:26:06.298154 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.298179 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:40137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 08:26:06.298313 7019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.188:40137: Network error: Client connection negotiation failed: client connection to 127.25.127.188:40137: connect: Connection refused (error 111)
I20260619 08:26:06.299688 7101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:33855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:33187,127.25.127.189:44381,127.25.127.188:40137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.188.keytab
--principal=kudu/127.25.127.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.188:40137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.127.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:06.299898 7101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:06.300073 7101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:06.304879 7109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:06.304937 7106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:06.304960 7101 server_base.cc:1034] running on GCE node
W20260619 08:26:06.304936 7107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:06.305348 7101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:06.305703 7101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:06.306922 7101 hybrid_clock.cc:648] HybridClock initialized: now 1781857566306900 us; error 40 us; skew 500 ppm
Jun 19 08:26:06 dist-test-slave-5rhg krb5kdc[6937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857566, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:06.309479 7101 init.cc:377] Logged in from keytab as kudu/127.25.127.188@KRBTEST.COM (short username kudu)
I20260619 08:26:06.309937 7101 webserver.cc:460] Webserver started at http://127.25.127.188:44773/ using document root <none> and password file <none>
I20260619 08:26:06.310171 7101 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:06.310241 7101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:06.310410 7101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:26:06.311560 7101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-2/data/instance:
uuid: "dced949850564955b3ca142749b8964e"
format_stamp: "Formatted at 2026-06-19 08:26:06 on dist-test-slave-5rhg"
server_key: "6d35f4c7fdad2bdb24f0b56192bd1cfd"
server_key_iv: "f04a660709a128158e4eee5573415af5"
server_key_version: "encryptionkey@0"
I20260619 08:26:06.311918 7101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-2/wal/instance:
uuid: "dced949850564955b3ca142749b8964e"
format_stamp: "Formatted at 2026-06-19 08:26:06 on dist-test-slave-5rhg"
server_key: "6d35f4c7fdad2bdb24f0b56192bd1cfd"
server_key_iv: "f04a660709a128158e4eee5573415af5"
server_key_version: "encryptionkey@0"
I20260619 08:26:06.313354 7101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:26:06.314067 7116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:06.314239 7101 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:26:06.314347 7101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-2/wal
uuid: "dced949850564955b3ca142749b8964e"
format_stamp: "Formatted at 2026-06-19 08:26:06 on dist-test-slave-5rhg"
server_key: "6d35f4c7fdad2bdb24f0b56192bd1cfd"
server_key_iv: "f04a660709a128158e4eee5573415af5"
server_key_version: "encryptionkey@0"
I20260619 08:26:06.314422 7101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:06.325878 7094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } attempt: 2
I20260619 08:26:06.326356 7096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.326223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:40137 (local address 127.0.0.1:48704)
0619 08:26:06.326284 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.326325 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:40137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 08:26:06.326684 7096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.326605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:40137 (local address 127.0.0.1:48716)
0619 08:26:06.326645 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.326669 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:40137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 08:26:06.326771 7094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.188:40137: Network error: Client connection negotiation failed: client connection to 127.25.127.188:40137: connect: Connection refused (error 111)
I20260619 08:26:06.334002 7101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:06.336421 7101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:06.336534 7101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:06.340454 7101 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.188:40137
I20260619 08:26:06.340518 7169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.188:40137 every 8 connection(s)
I20260619 08:26:06.340919 7101 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-2/data/info.pb
I20260619 08:26:06.341830 7170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:06.343447 7170 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:26:06.347720 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 7101
I20260619 08:26:06.347926 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781857412505079-26110-0/minicluster-data/master-2/wal/instance
I20260619 08:26:06.348129 26110 external_mini_cluster.cc:1428] Setting key 471fdeedd78701f10eda9f4bb89736d7
Jun 19 08:26:06 dist-test-slave-5rhg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857566, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.188@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:06.352855 7022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.344191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54270 (local address 127.25.127.190:33187)
0619 08:26:06.344236 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:06.344243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:06.344478 (+ 235us) server_negotiation.cc:407] Connection header received
0619 08:26:06.344916 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:06.344917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:06.344926 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:06.344983 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:06.345624 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.346103 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.347059 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.347281 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.347345 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.349795 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:06.349797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:06.349797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:06.349811 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:06.351219 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.351636 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.351637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.351638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.351700 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.352074 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.352075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.352075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.352240 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:06.352295 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:06.352685 (+ 390us) server_negotiation.cc:299] Negotiation successful
0619 08:26:06.352773 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:26:06.353088 7172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.344133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:33187 (local address 127.0.0.1:54270)
0619 08:26:06.344346 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.344391 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:06.344832 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:06.345048 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:06.345051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:06.345296 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:06.345509 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:06.345513 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.346188 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.346189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:06.346975 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:06.346977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.347390 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.347391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:06.347576 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.348226 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:06.348240 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:06.349693 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:06.351320 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:06.351328 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:06.351330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:06.351558 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:06.351798 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:06.351805 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:06.351805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:06.351990 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:06.352319 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:06.352327 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:06.352517 (+ 190us) client_negotiation.cc:769] Sending connection context
0619 08:26:06.352652 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 08:26:06.352832 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":92,"threads_started":1}
I20260619 08:26:06.353780 7170 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:26:06 dist-test-slave-5rhg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.188@KRBTEST.COM
I20260619 08:26:06.357911 7174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.348587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48732 (local address 127.25.127.188:40137)
0619 08:26:06.348864 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:06.348868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:06.348884 (+ 16us) server_negotiation.cc:407] Connection header received
0619 08:26:06.348901 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:06.348903 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:06.348918 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:06.348999 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:06.349645 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.350339 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.351083 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.351404 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.351486 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.354496 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:06.354504 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:06.354505 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:06.354511 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:06.356303 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.356781 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.356784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.356785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.356907 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.357318 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.357319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.357320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.357500 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:06.357678 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:06.357727 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 08:26:06.357836 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":71,"threads_started":1}
Jun 19 08:26:06 dist-test-slave-5rhg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857566, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.188@KRBTEST.COM for kudu/127.25.127.189@KRBTEST.COM
I20260619 08:26:06.361227 7172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.354108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:44381 (local address 127.0.0.1:53896)
0619 08:26:06.354153 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.354162 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:06.354200 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:06.354387 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:06.354387 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:06.354517 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:06.354565 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:06.354567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.355406 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.355407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:06.356269 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:06.356272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.356761 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.356762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:06.356845 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.357165 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:06.357169 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:06.358615 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:06.360374 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:06.360375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:06.360376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:06.360530 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:06.360729 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:06.360730 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:06.360730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:06.360776 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:06.361022 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:06.361022 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:06.361054 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 08:26:06.361102 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 08:26:06.361165 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 08:26:06.361308 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.354191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53896 (local address 127.25.127.189:44381)
0619 08:26:06.354256 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:06.354258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:06.354267 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:26:06.354292 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:06.354293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:06.354302 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:06.354368 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:06.354646 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.355334 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.356372 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.356675 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.356741 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.358717 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:06.358718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:06.358718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:06.358726 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:06.360304 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.360603 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.360603 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.360604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.360664 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.360830 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.360831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.360831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.360965 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:06.361026 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:06.361124 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 08:26:06.361190 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 08:26:06.361626 7170 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:26:06 dist-test-slave-5rhg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
Jun 19 08:26:06 dist-test-slave-5rhg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857566, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.188@KRBTEST.COM for kudu/127.25.127.188@KRBTEST.COM
I20260619 08:26:06.366812 7022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.358853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54272 (local address 127.25.127.190:33187)
0619 08:26:06.358899 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:06.358901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:06.358909 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:26:06.358931 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:06.358932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:06.358942 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:06.359000 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:06.359568 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.360349 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.361317 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.361616 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.361679 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.363862 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:06.363863 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:06.363863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:06.363870 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:06.365628 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.365956 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.365957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.365957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.366011 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.366223 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.366224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.366224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.366335 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:06.366542 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:06.366578 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 08:26:06.366650 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:26:06.368686 7172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.361918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:40137 (local address 127.0.0.1:48738)
0619 08:26:06.361989 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.361998 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:06.362058 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:06.362175 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:06.362176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:06.362323 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:06.362361 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:06.362363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.362953 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.362954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:06.363472 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:06.363494 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.363829 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.363830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:06.363878 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.364289 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:06.364294 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:06.365546 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:06.367493 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:06.367493 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:06.367494 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:06.367719 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:06.367946 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:06.367947 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:06.367947 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:06.368001 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:06.368287 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:06.368288 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:06.368349 (+ 61us) client_negotiation.cc:769] Sending connection context
0619 08:26:06.368429 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 08:26:06.368544 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 08:26:06.368789 7174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.361950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48738 (local address 127.25.127.188:40137)
0619 08:26:06.361988 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:06.361990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:06.362028 (+ 38us) server_negotiation.cc:407] Connection header received
0619 08:26:06.362105 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:06.362106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:06.362118 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:06.362173 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:06.362433 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.362858 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.363575 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.363761 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.363831 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.365648 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:06.365649 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:06.365650 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:06.365656 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:06.367384 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.367799 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.367800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.367801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.367861 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.368063 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.368064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.368064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.368223 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:06.368286 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:06.368451 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 08:26:06.368618 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":19}
I20260619 08:26:06.370437 7170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e: Bootstrap starting.
I20260619 08:26:06.371307 7170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:06.371609 7170 log.cc:824] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:06.372579 7170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e: No bootstrap required, opened a new log
Jun 19 08:26:06 dist-test-slave-5rhg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.189@KRBTEST.COM
I20260619 08:26:06.373478 7019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } attempt: 2
I20260619 08:26:06.374745 7170 raft_consensus.cc:348] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } }
I20260619 08:26:06.374903 7170 raft_consensus.cc:374] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:06.374953 7170 raft_consensus.cc:729] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dced949850564955b3ca142749b8964e, State: Initialized, Role: FOLLOWER
I20260619 08:26:06.375118 7170 consensus_queue.cc:260] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } }
I20260619 08:26:06.375394 7176 sys_catalog.cc:455] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } } }
I20260619 08:26:06.375461 7170 sys_catalog.cc:564] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:26:06.375536 7176 sys_catalog.cc:458] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:26:06.376451 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.367473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53908 (local address 127.25.127.189:44381)
0619 08:26:06.367520 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:06.367522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:06.367530 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:26:06.367553 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:06.367555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:06.367565 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:06.367633 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:06.368212 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.369367 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.370259 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.370486 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.370577 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.373247 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:06.373248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:06.373249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:06.373257 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:06.374907 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.375379 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.375380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.375380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.375441 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.375659 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.375660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.375660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.375787 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:06.375981 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:06.376287 (+ 306us) server_negotiation.cc:299] Negotiation successful
0619 08:26:06.376383 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260619 08:26:06.377703 7187 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:26:06.377789 7187 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:26:06.378561 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
Jun 19 08:26:06 dist-test-slave-5rhg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857565, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for kudu/127.25.127.188@KRBTEST.COM
I20260619 08:26:06.378198 7101 master_runner.cc:186] Error getting master registration for 127.25.127.190:33187: OK, instance_id { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" instance_seqno: 1781857566028650 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 08:26:06.381759 7101 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 08:26:06.382676 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.373761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:40137 (local address 127.0.0.1:48748)
0619 08:26:06.373823 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.373835 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:06.373874 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:06.374074 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:06.374076 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:06.374231 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:06.374298 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:06.374301 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.375926 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.375928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:06.376805 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:06.376809 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.377368 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.377369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:06.377440 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.377848 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:06.377853 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:06.379391 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:06.381467 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:06.381468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:06.381469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:06.381703 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:06.381935 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:06.381935 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:06.381936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:06.382000 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:06.382334 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:06.382335 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:06.382387 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 08:26:06.382455 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 08:26:06.382558 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 08:26:06.383061 7174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.373878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48748 (local address 127.25.127.188:40137)
0619 08:26:06.373953 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:06.373955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:06.373962 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:26:06.373981 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:06.373982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:06.373992 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:06.374042 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:06.375028 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.375838 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.376880 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.377197 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.377304 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.379490 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:06.379491 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:06.379492 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:06.379498 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:06.381365 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.381790 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.381791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.381791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.381855 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.382069 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.382070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.382070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.382226 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:06.382303 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:06.382877 (+ 574us) server_negotiation.cc:299] Negotiation successful
0619 08:26:06.382966 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 08:26:06.385193 7019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90: Bootstrap starting.
I20260619 08:26:06.386016 7019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:06.386368 7019 log.cc:824] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:06.387531 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.379946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53918 (local address 127.25.127.189:44381)
0619 08:26:06.379982 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:06.379984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:06.380261 (+ 277us) server_negotiation.cc:407] Connection header received
0619 08:26:06.380363 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:06.380364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:06.380375 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:06.380406 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:06.381219 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.381981 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.382863 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.383155 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.383225 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.384575 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:06.384577 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:06.384577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:06.384583 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:06.386195 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.386551 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.386552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.386553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.386608 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.386782 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.386783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.386783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.386915 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:06.387004 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:06.387329 (+ 325us) server_negotiation.cc:299] Negotiation successful
0619 08:26:06.387405 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:26:06.389487 7019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90: No bootstrap required, opened a new log
I20260619 08:26:06.391297 7022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.380176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54274 (local address 127.25.127.190:33187)
0619 08:26:06.380217 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:06.380219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:06.381373 (+ 1154us) server_negotiation.cc:407] Connection header received
0619 08:26:06.381463 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:06.381464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:06.381473 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:06.381500 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:06.384283 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.385269 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.386593 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.386887 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.386930 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.388447 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:06.388449 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:06.388449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:06.388457 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:06.390056 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.390433 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.390434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.390434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.390488 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.390690 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.390690 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.390691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.390823 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:06.390901 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:06.391125 (+ 224us) server_negotiation.cc:299] Negotiation successful
0619 08:26:06.391199 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:26:06.392184 7019 raft_consensus.cc:348] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } }
I20260619 08:26:06.392647 7019 raft_consensus.cc:374] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:06.392706 7019 raft_consensus.cc:729] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f810b83bb28c4f9082ad2ee9f10e8a90, State: Initialized, Role: FOLLOWER
I20260619 08:26:06.392846 7019 consensus_queue.cc:260] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } }
I20260619 08:26:06.393216 7019 sys_catalog.cc:564] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:26:06.393179 7198 sys_catalog.cc:455] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } } }
I20260619 08:26:06.393268 7198 sys_catalog.cc:458] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:26:06.395840 7195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.380782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48750 (local address 127.25.127.188:40137)
0619 08:26:06.381275 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:06.381277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:06.382175 (+ 898us) server_negotiation.cc:407] Connection header received
0619 08:26:06.383707 (+ 1532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:06.383708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:06.384809 (+ 1101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:06.384877 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:06.388354 (+ 3477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.389169 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.390159 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.390504 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.390543 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.391980 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:06.391981 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:06.391982 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:06.391989 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:06.394342 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.394701 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.394703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.394703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.394761 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.394977 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.394978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.394978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.395106 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:06.395212 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:06.395631 (+ 419us) server_negotiation.cc:299] Negotiation successful
0619 08:26:06.395715 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":74,"threads_started":1}
W20260619 08:26:06.395867 7209 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:26:06.396478 7209 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:26:06.396384 6950 master_runner.cc:186] Error getting master registration for 127.25.127.189:44381: OK, instance_id { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" instance_seqno: 1781857566183238 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 08:26:06.398566 6950 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 08:26:06.417877 7094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } attempt: 3
Jun 19 08:26:06 dist-test-slave-5rhg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857566, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.189@KRBTEST.COM for kudu/127.25.127.188@KRBTEST.COM
I20260619 08:26:06.425390 7195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.418279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48764 (local address 127.25.127.188:40137)
0619 08:26:06.418322 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:06.418324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:06.418332 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:26:06.418406 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:06.418407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:06.418419 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:06.418491 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:06.418843 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.419381 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.420184 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.420455 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.420552 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.422718 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:06.422719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:06.422720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:06.422727 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:06.424266 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.424614 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.424616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.424616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.424675 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:06.424871 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:06.424872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:06.424872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:06.424987 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:06.425057 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:06.425215 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 08:26:06.425290 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:26:06.425391 7096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:06.418228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:40137 (local address 127.0.0.1:48764)
0619 08:26:06.418283 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:06.418296 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:06.418341 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:06.418497 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:06.418498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:06.418674 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:06.418730 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:06.418731 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.419464 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.419465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:06.420011 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:06.420013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:06.420551 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:06.420552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:06.420614 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:06.421141 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:06.421147 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:06.422631 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:06.424367 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:06.424368 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:06.424368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:06.424541 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:06.424742 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:06.424743 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:06.424743 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:06.424806 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:06.425061 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:06.425062 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:06.425106 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 08:26:06.425188 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 08:26:06.425289 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 08:26:06.427099 7094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d: Bootstrap starting.
I20260619 08:26:06.427757 7094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:06.428072 7094 log.cc:824] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:06.428893 7094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d: No bootstrap required, opened a new log
I20260619 08:26:06.430452 7094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } }
I20260619 08:26:06.430585 7094 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:06.430630 7094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc79d4f00da44bf878c14c5608db92d, State: Initialized, Role: FOLLOWER
I20260619 08:26:06.430785 7094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } }
I20260619 08:26:06.431275 7094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:26:06.432947 7212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } } }
I20260619 08:26:06.433036 7212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d [sys.catalog]: This master's current role is: FOLLOWER
W20260619 08:26:06.433321 7224 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:26:06.433413 7224 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:26:06.435599 7026 master_runner.cc:263] No leader master found from master 5dc79d4f00da44bf878c14c5608db92d
I20260619 08:26:06.435642 7026 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 08:26:06.442646 7176 raft_consensus.cc:482] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:26:06.442742 7176 raft_consensus.cc:504] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } }
I20260619 08:26:06.442999 7176 leader_election.cc:288] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f810b83bb28c4f9082ad2ee9f10e8a90 (127.25.127.190:33187), 5dc79d4f00da44bf878c14c5608db92d (127.25.127.189:44381)
I20260619 08:26:06.443641 7068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dced949850564955b3ca142749b8964e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc79d4f00da44bf878c14c5608db92d" is_pre_election: true
I20260619 08:26:06.443802 7068 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dced949850564955b3ca142749b8964e in term 0.
I20260619 08:26:06.444093 7118 leader_election.cc:302] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dc79d4f00da44bf878c14c5608db92d, dced949850564955b3ca142749b8964e; no voters:
I20260619 08:26:06.444936 6993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dced949850564955b3ca142749b8964e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" is_pre_election: true
I20260619 08:26:06.445070 6993 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dced949850564955b3ca142749b8964e in term 0.
I20260619 08:26:06.445336 7176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:26:06.445400 7176 raft_consensus.cc:482] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:26:06.445423 7176 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:06.446306 7176 raft_consensus.cc:504] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } }
I20260619 08:26:06.446417 7176 leader_election.cc:288] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [CANDIDATE]: Term 1 election: Requested vote from peers f810b83bb28c4f9082ad2ee9f10e8a90 (127.25.127.190:33187), 5dc79d4f00da44bf878c14c5608db92d (127.25.127.189:44381)
I20260619 08:26:06.446763 7068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dced949850564955b3ca142749b8964e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc79d4f00da44bf878c14c5608db92d"
I20260619 08:26:06.446842 7068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:06.447683 7068 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dced949850564955b3ca142749b8964e in term 1.
I20260619 08:26:06.447865 7118 leader_election.cc:302] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dc79d4f00da44bf878c14c5608db92d, dced949850564955b3ca142749b8964e; no voters:
I20260619 08:26:06.448000 6993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dced949850564955b3ca142749b8964e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90"
I20260619 08:26:06.448074 6993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:06.449011 6993 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dced949850564955b3ca142749b8964e in term 1.
I20260619 08:26:06.449241 7176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:26:06.449405 7176 raft_consensus.cc:686] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [term 1 LEADER]: Becoming Leader. State: Replica: dced949850564955b3ca142749b8964e, State: Running, Role: LEADER
I20260619 08:26:06.449487 7176 consensus_queue.cc:237] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } }
I20260619 08:26:06.449934 7176 sys_catalog.cc:455] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dced949850564955b3ca142749b8964e. Latest consensus state: current_term: 1 leader_uuid: "dced949850564955b3ca142749b8964e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } } }
I20260619 08:26:06.450001 7176 sys_catalog.cc:458] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [sys.catalog]: This master's current role is: LEADER
I20260619 08:26:06.450299 7228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:26:06.450534 7228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:26:06.452759 7068 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d [term 1 FOLLOWER]: Refusing update from remote peer dced949850564955b3ca142749b8964e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:26:06.453028 6993 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90 [term 1 FOLLOWER]: Refusing update from remote peer dced949850564955b3ca142749b8964e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:26:06.453474 7176 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:26:06.453619 7176 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:26:06.454968 7198 sys_catalog.cc:455] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dced949850564955b3ca142749b8964e. Latest consensus state: current_term: 1 leader_uuid: "dced949850564955b3ca142749b8964e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } } }
I20260619 08:26:06.455051 7198 sys_catalog.cc:458] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:26:06.456103 7212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d [sys.catalog]: SysCatalogTable state changed. Reason: New leader dced949850564955b3ca142749b8964e. Latest consensus state: current_term: 1 leader_uuid: "dced949850564955b3ca142749b8964e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } } }
I20260619 08:26:06.456228 7212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:26:06.458709 7176 sys_catalog.cc:455] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dced949850564955b3ca142749b8964e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } } }
I20260619 08:26:06.458789 7176 sys_catalog.cc:458] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [sys.catalog]: This master's current role is: LEADER
I20260619 08:26:06.459370 7212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dced949850564955b3ca142749b8964e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } } }
I20260619 08:26:06.459457 7212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:26:06.459611 7229 mvcc.cc:205] Tried to move back new op lower bound from 7298488592187387904 to 7298488592177647616. Current Snapshot: MvccSnapshot[applied={T|T < 7298488592187387904}]
I20260619 08:26:06.460085 7176 sys_catalog.cc:455] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dced949850564955b3ca142749b8964e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } } }
I20260619 08:26:06.460211 7176 sys_catalog.cc:458] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e [sys.catalog]: This master's current role is: LEADER
I20260619 08:26:06.460601 7198 sys_catalog.cc:455] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dced949850564955b3ca142749b8964e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f810b83bb28c4f9082ad2ee9f10e8a90" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33187 } } peers { permanent_uuid: "5dc79d4f00da44bf878c14c5608db92d" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 44381 } } peers { permanent_uuid: "dced949850564955b3ca142749b8964e" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 40137 } } }
I20260619 08:26:06.460680 7198 sys_catalog.cc:458] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:26:06.461303 7230 mvcc.cc:205] Tried to move back new op lower bound from 7298488592187387904 to 7298488592177647616. Current Snapshot: MvccSnapshot[applied={T|T < 7298488592187387904}]
I20260619 08:26:06.461699 7232 mvcc.cc:205] Tried to move back new op lower bound from 7298488592187387904 to 7298488592177647616. Current Snapshot: MvccSnapshot[applied={T|T < 7298488592187387904}]
I20260619 08:26:06.462349 7228 catalog_manager.cc:1347] Generated new cluster ID: 076b208af6374ad4abe9449de4c30072
I20260619 08:26:06.462391 7228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:26:06.489352 7228 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:26:06.490253 7228 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:26:06.496979 7228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dced949850564955b3ca142749b8964e: Generated new TSK 0
I20260619 08:26:06.497304 7228 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:26:07.397032 7209 catalog_manager.cc:1259] Loaded cluster ID: 076b208af6374ad4abe9449de4c30072
I20260619 08:26:07.397106 7209 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90: loading cluster ID for follower catalog manager: success
I20260619 08:26:07.397974 7209 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90: acquiring CA information for follower catalog manager: success
I20260619 08:26:07.398298 7209 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f810b83bb28c4f9082ad2ee9f10e8a90: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 08:26:07.433883 7224 catalog_manager.cc:1259] Loaded cluster ID: 076b208af6374ad4abe9449de4c30072
I20260619 08:26:07.433957 7224 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d: loading cluster ID for follower catalog manager: success
I20260619 08:26:07.434773 7224 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d: acquiring CA information for follower catalog manager: success
I20260619 08:26:07.435122 7224 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5dc79d4f00da44bf878c14c5608db92d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 08:26:07.551851 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 6950
I20260619 08:26:07.557483 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 7026
I20260619 08:26:07.562808 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 7101
2026-06-19T08:26:07Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2825 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:26:07 dist-test-slave-5rhg krb5kdc[7244](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:26:07 dist-test-slave-5rhg krb5kdc[7244](info): set up 2 sockets
Jun 19 08:26:07 dist-test-slave-5rhg krb5kdc[7244](info): commencing operation
krb5kdc: starting...
W20260619 08:26:08.645073 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:26:08 dist-test-slave-5rhg krb5kdc[7244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:26:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:26:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:26:08.767122 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:40933
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:37913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:40933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:26:08.850907 7257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:08.851140 7257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:08.851181 7257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:08.852414 7257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:26:08.852470 7257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:08.852496 7257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:08.852540 7257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:26:08.852571 7257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:26:08.854036 7257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:37913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:40933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:40933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:08.854288 7257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:08.854519 7257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:08.859454 7262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:08.859438 7263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:08.859437 7265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:08.859907 7257 server_base.cc:1034] running on GCE node
I20260619 08:26:08.860193 7257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:08.860498 7257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:08.861667 7257 hybrid_clock.cc:648] HybridClock initialized: now 1781857568861647 us; error 47 us; skew 500 ppm
Jun 19 08:26:08 dist-test-slave-5rhg krb5kdc[7244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857568, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:08.864122 7257 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:26:08.864601 7257 webserver.cc:460] Webserver started at http://127.25.127.190:43627/ using document root <none> and password file <none>
I20260619 08:26:08.864850 7257 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:08.864919 7257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:08.865069 7257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:26:08.866174 7257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "91ae8f09772548dd85fb791576c70e39"
format_stamp: "Formatted at 2026-06-19 08:26:08 on dist-test-slave-5rhg"
server_key: "b5a86aef27d7c08ae11b83b1b24c1cd3"
server_key_iv: "4a00439e024309edf2160ac5e09f9f23"
server_key_version: "encryptionkey@0"
I20260619 08:26:08.866539 7257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "91ae8f09772548dd85fb791576c70e39"
format_stamp: "Formatted at 2026-06-19 08:26:08 on dist-test-slave-5rhg"
server_key: "b5a86aef27d7c08ae11b83b1b24c1cd3"
server_key_iv: "4a00439e024309edf2160ac5e09f9f23"
server_key_version: "encryptionkey@0"
I20260619 08:26:08.867904 7257 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:26:08.868773 7272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:08.868956 7257 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:26:08.869076 7257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "91ae8f09772548dd85fb791576c70e39"
format_stamp: "Formatted at 2026-06-19 08:26:08 on dist-test-slave-5rhg"
server_key: "b5a86aef27d7c08ae11b83b1b24c1cd3"
server_key_iv: "4a00439e024309edf2160ac5e09f9f23"
server_key_version: "encryptionkey@0"
I20260619 08:26:08.869176 7257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:08.879390 7257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:08.881449 7257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:08.881631 7257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:08.886113 7257 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:40933
I20260619 08:26:08.886143 7324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:40933 every 8 connection(s)
I20260619 08:26:08.886534 7257 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:26:08.887450 7325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:08.889889 7325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39: Bootstrap starting.
I20260619 08:26:08.890492 7325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:08.890787 7325 log.cc:824] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:08.891498 7325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39: No bootstrap required, opened a new log
I20260619 08:26:08.892000 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 7257
I20260619 08:26:08.892139 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:26:08.892407 26110 external_mini_cluster.cc:1428] Setting key 9f8240c50dfdeaa0cb31a99b986636f9
I20260619 08:26:08.894591 7325 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91ae8f09772548dd85fb791576c70e39" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40933 } }
I20260619 08:26:08.894759 7325 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:08.894825 7325 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ae8f09772548dd85fb791576c70e39, State: Initialized, Role: FOLLOWER
I20260619 08:26:08.894982 7325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91ae8f09772548dd85fb791576c70e39" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40933 } }
I20260619 08:26:08.895084 7325 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:26:08.895130 7325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:26:08.895174 7325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:08.895761 7325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91ae8f09772548dd85fb791576c70e39" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40933 } }
I20260619 08:26:08.895915 7325 leader_election.cc:302] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91ae8f09772548dd85fb791576c70e39; no voters:
I20260619 08:26:08.896121 7325 leader_election.cc:288] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:26:08.896178 7330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:26:08.896339 7330 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39 [term 1 LEADER]: Becoming Leader. State: Replica: 91ae8f09772548dd85fb791576c70e39, State: Running, Role: LEADER
I20260619 08:26:08.896454 7325 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:26:08.896485 7330 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91ae8f09772548dd85fb791576c70e39" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40933 } }
I20260619 08:26:08.896829 7330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91ae8f09772548dd85fb791576c70e39. Latest consensus state: current_term: 1 leader_uuid: "91ae8f09772548dd85fb791576c70e39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91ae8f09772548dd85fb791576c70e39" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40933 } } }
I20260619 08:26:08.896844 7331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91ae8f09772548dd85fb791576c70e39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91ae8f09772548dd85fb791576c70e39" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40933 } } }
I20260619 08:26:08.896958 7330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39 [sys.catalog]: This master's current role is: LEADER
I20260619 08:26:08.897230 7331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39 [sys.catalog]: This master's current role is: LEADER
I20260619 08:26:08.897431 7339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:26:08.898288 7339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:26:08.900027 7339 catalog_manager.cc:1347] Generated new cluster ID: 8116498a69f148fa992e2ddfbf0217c8
I20260619 08:26:08.900087 7339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:26:08 dist-test-slave-5rhg krb5kdc[7244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:08.904886 7329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:08.892965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48828 (local address 127.25.127.190:40933)
0619 08:26:08.893142 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:08.893153 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:08.893210 (+ 57us) server_negotiation.cc:407] Connection header received
0619 08:26:08.894524 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:08.894533 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:08.894594 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:08.894768 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:08.895541 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:08.896852 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:08.898095 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:08.898691 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:08.898818 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:08.901500 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:08.901504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:08.901505 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:08.901518 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:08.903242 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:08.903644 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:08.903647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:08.903647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:08.903739 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:08.903969 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:08.903970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:08.903970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:08.904297 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:08.904476 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:08.904557 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 08:26:08.904702 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260619 08:26:08.906005 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:26:08.920508 7339 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:26:08.921280 7339 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:26:08.923992 7339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91ae8f09772548dd85fb791576c70e39: Generated new TSK 0
I20260619 08:26:08.924218 7339 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:26:08.960111 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 7257
2026-06-19T08:26:08Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1395 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:26:09 dist-test-slave-5rhg krb5kdc[7353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:26:09 dist-test-slave-5rhg krb5kdc[7353](info): set up 2 sockets
Jun 19 08:26:09 dist-test-slave-5rhg krb5kdc[7353](info): commencing operation
krb5kdc: starting...
W20260619 08:26:10.037410 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:26:10 dist-test-slave-5rhg krb5kdc[7353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:26:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:26:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:26:10.159449 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:34081
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:34081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:26:10.240789 7366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:10.241003 7366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:10.241025 7366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:10.242223 7366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:26:10.242267 7366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:10.242282 7366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:10.242295 7366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:26:10.242313 7366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:26:10.243789 7366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:34081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:34081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:10.244046 7366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:10.244297 7366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:10.249223 7371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:10.249254 7374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:10.249343 7366 server_base.cc:1034] running on GCE node
W20260619 08:26:10.249230 7372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:10.249740 7366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:10.250066 7366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:10.251255 7366 hybrid_clock.cc:648] HybridClock initialized: now 1781857570251248 us; error 36 us; skew 500 ppm
Jun 19 08:26:10 dist-test-slave-5rhg krb5kdc[7353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857570, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:10.253767 7366 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:26:10.254199 7366 webserver.cc:460] Webserver started at http://127.25.127.190:37101/ using document root <none> and password file <none>
I20260619 08:26:10.254452 7366 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:10.254521 7366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:10.254675 7366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:26:10.255776 7366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "883181651454435081075e99cad82cb8"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "6583794567092c68f15437a81984576f"
server_key_iv: "27c93fdb860ee77ec60a3905c318c9fe"
server_key_version: "encryptionkey@0"
I20260619 08:26:10.256201 7366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "883181651454435081075e99cad82cb8"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "6583794567092c68f15437a81984576f"
server_key_iv: "27c93fdb860ee77ec60a3905c318c9fe"
server_key_version: "encryptionkey@0"
I20260619 08:26:10.257615 7366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:26:10.258384 7381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:10.258571 7366 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:10.258672 7366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "883181651454435081075e99cad82cb8"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "6583794567092c68f15437a81984576f"
server_key_iv: "27c93fdb860ee77ec60a3905c318c9fe"
server_key_version: "encryptionkey@0"
I20260619 08:26:10.258760 7366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:10.289811 7366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:10.291872 7366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:10.292057 7366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:10.296042 7366 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:34081
I20260619 08:26:10.296109 7433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:34081 every 8 connection(s)
I20260619 08:26:10.296512 7366 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:26:10.297506 7434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:10.299865 7434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8: Bootstrap starting.
I20260619 08:26:10.300521 7434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:10.300829 7434 log.cc:824] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:10.301523 7434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8: No bootstrap required, opened a new log
I20260619 08:26:10.303138 7434 raft_consensus.cc:348] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883181651454435081075e99cad82cb8" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 34081 } }
I20260619 08:26:10.303278 7434 raft_consensus.cc:374] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:10.303301 7434 raft_consensus.cc:729] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 883181651454435081075e99cad82cb8, State: Initialized, Role: FOLLOWER
I20260619 08:26:10.303483 7434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883181651454435081075e99cad82cb8" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 34081 } }
I20260619 08:26:10.303545 7434 raft_consensus.cc:388] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:26:10.303586 7434 raft_consensus.cc:482] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:26:10.303637 7434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:10.304522 7434 raft_consensus.cc:504] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883181651454435081075e99cad82cb8" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 34081 } }
I20260619 08:26:10.304634 7434 leader_election.cc:302] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 883181651454435081075e99cad82cb8; no voters:
I20260619 08:26:10.304817 7434 leader_election.cc:288] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:26:10.304876 7437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:26:10.305090 7437 raft_consensus.cc:686] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 1 LEADER]: Becoming Leader. State: Replica: 883181651454435081075e99cad82cb8, State: Running, Role: LEADER
I20260619 08:26:10.305178 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 7366
I20260619 08:26:10.305088 7434 sys_catalog.cc:564] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:26:10.305298 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:26:10.305320 7437 consensus_queue.cc:237] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883181651454435081075e99cad82cb8" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 34081 } }
I20260619 08:26:10.305507 26110 external_mini_cluster.cc:1428] Setting key 4fa9536f4d230642db7e1d8233ae7d45
I20260619 08:26:10.305722 7438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "883181651454435081075e99cad82cb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883181651454435081075e99cad82cb8" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 34081 } } }
I20260619 08:26:10.305876 7438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [sys.catalog]: This master's current role is: LEADER
I20260619 08:26:10.305734 7439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 883181651454435081075e99cad82cb8. Latest consensus state: current_term: 1 leader_uuid: "883181651454435081075e99cad82cb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883181651454435081075e99cad82cb8" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 34081 } } }
I20260619 08:26:10.306128 7442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:26:10.306178 7439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [sys.catalog]: This master's current role is: LEADER
I20260619 08:26:10.306874 7442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:26:10.308486 7442 catalog_manager.cc:1347] Generated new cluster ID: ebf317f7bb724fcabc1edc6c8b60bcb1
I20260619 08:26:10.308573 7442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:26:10 dist-test-slave-5rhg krb5kdc[7353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:10.317046 7442 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:26:10.317714 7442 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:26:10.318578 7444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:10.305980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42958 (local address 127.25.127.190:34081)
0619 08:26:10.306263 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:10.306274 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:10.307330 (+ 1056us) server_negotiation.cc:407] Connection header received
0619 08:26:10.307490 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:10.307497 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:10.307550 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:10.307696 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:10.308550 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.309917 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.310987 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.311656 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.311785 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:10.314609 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:10.314613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:10.314613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:10.314625 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:10.316796 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:10.317215 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:10.317218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:10.317218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:10.317321 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:10.317554 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:10.317555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:10.317555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:10.317833 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:10.317995 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:10.318115 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 08:26:10.318360 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":63,"threads_started":1}
I20260619 08:26:10.325232 7442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8: Generated new TSK 0
I20260619 08:26:10.325440 7442 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
I20260619 08:26:10.359193 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:34081
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:26:10.442669 7460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:10.442909 7460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:10.442934 7460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:10.444082 7460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:10.444120 7460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:10.444165 7460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:26:10.445657 7460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:34081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:10.445869 7460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:10.446033 7460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:10.451153 7466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:10.451179 7465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:10.451210 7460 server_base.cc:1034] running on GCE node
W20260619 08:26:10.451153 7468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:10.451579 7460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:10.451785 7460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:10.452962 7460 hybrid_clock.cc:648] HybridClock initialized: now 1781857570452948 us; error 44 us; skew 500 ppm
Jun 19 08:26:10 dist-test-slave-5rhg krb5kdc[7353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857570, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:10.455412 7460 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:26:10.455855 7460 webserver.cc:460] Webserver started at http://127.25.127.129:33731/ using document root <none> and password file <none>
I20260619 08:26:10.456084 7460 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:10.456151 7460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:10.456331 7460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:26:10.457410 7460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "a800160590984b53bfe805a885538ec3"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "b8b201a0c74d7dd6dff6553889bfe231"
server_key_iv: "1447816804169109e55ad58b7c1dac2e"
server_key_version: "encryptionkey@0"
I20260619 08:26:10.457779 7460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "a800160590984b53bfe805a885538ec3"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "b8b201a0c74d7dd6dff6553889bfe231"
server_key_iv: "1447816804169109e55ad58b7c1dac2e"
server_key_version: "encryptionkey@0"
I20260619 08:26:10.459120 7460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:26:10.459810 7475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:10.460000 7460 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:26:10.460098 7460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "a800160590984b53bfe805a885538ec3"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "b8b201a0c74d7dd6dff6553889bfe231"
server_key_iv: "1447816804169109e55ad58b7c1dac2e"
server_key_version: "encryptionkey@0"
I20260619 08:26:10.460227 7460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:10.471061 7460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:10.473073 7460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:10.473234 7460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:10.473477 7460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:10.473793 7460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:26:10.473852 7460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:10.473894 7460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:26:10.473922 7460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:10.480194 7460 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:33441
I20260619 08:26:10.480218 7589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:33441 every 8 connection(s)
I20260619 08:26:10.480566 7460 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:26:10.484400 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 7460
I20260619 08:26:10.484535 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:26:10.484741 26110 external_mini_cluster.cc:1428] Setting key 92982b8aed6757fcf5dc7f12a395c81b
Jun 19 08:26:10 dist-test-slave-5rhg krb5kdc[7353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857570, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:10.490619 7444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:10.481329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:60911 (local address 127.25.127.190:34081)
0619 08:26:10.481428 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:10.481430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:10.481609 (+ 179us) server_negotiation.cc:407] Connection header received
0619 08:26:10.482152 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:10.482154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:10.482166 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:10.482223 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:10.483210 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.483717 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.484896 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.485134 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.485199 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:10.487442 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:10.487444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:10.487444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:10.487453 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:10.489137 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:10.489544 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:10.489546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:10.489546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:10.489625 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:10.489861 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:10.489862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:10.489863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:10.490033 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:10.490117 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:10.490362 (+ 245us) server_negotiation.cc:299] Negotiation successful
0619 08:26:10.490480 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 08:26:10.490667 7592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:10.481262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:60911)
0619 08:26:10.481459 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:10.481510 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:10.482047 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:10.482319 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:10.482327 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:10.482897 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:10.483100 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:10.483104 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.483810 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.483811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:10.484807 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:10.484810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.485205 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.485206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:10.485350 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:10.485933 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:10.485941 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:10.487340 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:10.489215 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:10.489222 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:10.489224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:10.489465 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:10.489691 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:10.489692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:10.489693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:10.489792 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:10.490105 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:10.490109 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:10.490227 (+ 118us) client_negotiation.cc:769] Sending connection context
0619 08:26:10.490322 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 08:26:10.490449 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":95,"threads_started":1}
I20260619 08:26:10.491338 7590 heartbeater.cc:347] Connected to a master server at 127.25.127.190:34081
I20260619 08:26:10.491423 7590 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:10.491643 7590 heartbeater.cc:511] Master 127.25.127.190:34081 requested a full tablet report, sending...
I20260619 08:26:10.492305 7398 ts_manager.cc:194] Registered new tserver with Master: a800160590984b53bfe805a885538ec3 (127.25.127.129:33441)
I20260619 08:26:10.492933 7398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:60911
WARNING: no policy specified for kudu/127.25.127.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
I20260619 08:26:10.510435 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:34081
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:26:10.593325 7595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:10.593585 7595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:10.593631 7595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:10.594821 7595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:10.594898 7595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:10.594945 7595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:26:10.596488 7595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:34081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:10.596797 7595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:10.597054 7595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:10.602041 7601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:10.602100 7603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:10.602172 7600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:10.602406 7595 server_base.cc:1034] running on GCE node
I20260619 08:26:10.602600 7595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:10.602846 7595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:10.604007 7595 hybrid_clock.cc:648] HybridClock initialized: now 1781857570604004 us; error 39 us; skew 500 ppm
Jun 19 08:26:10 dist-test-slave-5rhg krb5kdc[7353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857570, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:10.606734 7595 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:26:10.607170 7595 webserver.cc:460] Webserver started at http://127.25.127.130:40077/ using document root <none> and password file <none>
I20260619 08:26:10.607412 7595 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:10.607491 7595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:10.607647 7595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:26:10.608809 7595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "ade1c097ed4646dfa7d5cb6415591387"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "57291e2a88cb8ae433dd7309d1c98567"
server_key_iv: "217e891fb7a308d476fa47d8c98a04aa"
server_key_version: "encryptionkey@0"
I20260619 08:26:10.609179 7595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "ade1c097ed4646dfa7d5cb6415591387"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "57291e2a88cb8ae433dd7309d1c98567"
server_key_iv: "217e891fb7a308d476fa47d8c98a04aa"
server_key_version: "encryptionkey@0"
I20260619 08:26:10.610486 7595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:26:10.611231 7610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:10.611405 7595 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:26:10.611510 7595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "ade1c097ed4646dfa7d5cb6415591387"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "57291e2a88cb8ae433dd7309d1c98567"
server_key_iv: "217e891fb7a308d476fa47d8c98a04aa"
server_key_version: "encryptionkey@0"
I20260619 08:26:10.611588 7595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:10.635036 7595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:10.637562 7595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:10.637734 7595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:10.637976 7595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:10.638321 7595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:26:10.638376 7595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:10.638420 7595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:26:10.638448 7595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:10.644322 7595 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:42971
I20260619 08:26:10.644394 7723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:42971 every 8 connection(s)
I20260619 08:26:10.644727 7595 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:26:10.645673 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 7595
I20260619 08:26:10.645808 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:26:10.646020 26110 external_mini_cluster.cc:1428] Setting key 7d033400a2e1a0ce19f75923fbe3af4d
Jun 19 08:26:10 dist-test-slave-5rhg krb5kdc[7353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857570, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.127.131@KRBTEST.COM; defaulting to no policy
I20260619 08:26:10.654989 7444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:10.645473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:49993 (local address 127.25.127.190:34081)
0619 08:26:10.645558 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:10.645560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:10.645785 (+ 225us) server_negotiation.cc:407] Connection header received
0619 08:26:10.646473 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:10.646473 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:10.646482 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:10.646523 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:10.647843 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.648455 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.649525 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.649722 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.649850 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:10.652058 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:10.652060 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:10.652060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:10.652067 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:10.653537 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:10.653960 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:10.653961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:10.653962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:10.654027 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:10.654282 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:10.654283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:10.654283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:10.654421 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:10.654496 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:10.654808 (+ 312us) server_negotiation.cc:299] Negotiation successful
0619 08:26:10.654892 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 08:26:10.655184 7726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:10.645440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:49993)
0619 08:26:10.645634 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:10.645687 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:10.646374 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:10.646596 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:10.646600 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:10.647453 (+ 853us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:10.647746 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:10.647753 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.648557 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.648558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:10.649412 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:10.649414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.649862 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.649863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:10.650003 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:10.650568 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:10.650576 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:10.651955 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:10.653623 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:10.653632 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:10.653634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:10.653867 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:10.654095 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:10.654096 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:10.654097 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:10.654210 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:10.654500 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:10.654507 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:10.654649 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 08:26:10.654786 (+ 137us) client_negotiation.cc:240] Negotiation successful
0619 08:26:10.654953 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":96,"threads_started":1}
I20260619 08:26:10.655719 7724 heartbeater.cc:347] Connected to a master server at 127.25.127.190:34081
I20260619 08:26:10.655824 7724 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:10.655967 7724 heartbeater.cc:511] Master 127.25.127.190:34081 requested a full tablet report, sending...
I20260619 08:26:10.656384 7398 ts_manager.cc:194] Registered new tserver with Master: ade1c097ed4646dfa7d5cb6415591387 (127.25.127.130:42971)
I20260619 08:26:10.656769 7398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:49993
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
I20260619 08:26:10.671136 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:34081
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:26:10.751621 7729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:10.751855 7729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:10.751892 7729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:10.753121 7729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:10.753194 7729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:10.753240 7729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:26:10.754738 7729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:34081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:10.755004 7729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:10.755236 7729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:10.760174 7735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:10.760178 7734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:10.760368 7729 server_base.cc:1034] running on GCE node
W20260619 08:26:10.760190 7737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:10.760677 7729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:10.760924 7729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:10.762099 7729 hybrid_clock.cc:648] HybridClock initialized: now 1781857570762083 us; error 44 us; skew 500 ppm
Jun 19 08:26:10 dist-test-slave-5rhg krb5kdc[7353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857570, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:10.764832 7729 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:26:10.765210 7729 webserver.cc:460] Webserver started at http://127.25.127.131:45767/ using document root <none> and password file <none>
I20260619 08:26:10.765406 7729 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:10.765446 7729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:10.765533 7729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:26:10.766535 7729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "7135752d05f347399a2580f52c9abe22"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "223b70c3a1a46eac0840669fb1efefac"
server_key_iv: "ea7d9e6c76ccb3fa7e01974049f3b0c0"
server_key_version: "encryptionkey@0"
I20260619 08:26:10.766842 7729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "7135752d05f347399a2580f52c9abe22"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "223b70c3a1a46eac0840669fb1efefac"
server_key_iv: "ea7d9e6c76ccb3fa7e01974049f3b0c0"
server_key_version: "encryptionkey@0"
I20260619 08:26:10.768066 7729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:26:10.768877 7745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:10.769055 7729 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:26:10.769162 7729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "7135752d05f347399a2580f52c9abe22"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "223b70c3a1a46eac0840669fb1efefac"
server_key_iv: "ea7d9e6c76ccb3fa7e01974049f3b0c0"
server_key_version: "encryptionkey@0"
I20260619 08:26:10.769243 7729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:10.785267 7729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:10.787814 7729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:10.787988 7729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:10.788296 7729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:10.788636 7729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:26:10.788694 7729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:10.788735 7729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:26:10.788771 7729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:10.794905 7729 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:39281
I20260619 08:26:10.794951 7858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:39281 every 8 connection(s)
I20260619 08:26:10.795288 7729 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:26:10.795830 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 7729
I20260619 08:26:10.795943 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:26:10.796208 26110 external_mini_cluster.cc:1428] Setting key 08115ae98b8e4486226a4cb59bc5c586
Jun 19 08:26:10 dist-test-slave-5rhg krb5kdc[7353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857570, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:10.804912 7444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:10.796019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:40903 (local address 127.25.127.190:34081)
0619 08:26:10.796074 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:10.796077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:10.796356 (+ 279us) server_negotiation.cc:407] Connection header received
0619 08:26:10.796893 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:10.796894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:10.796907 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:10.796967 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:10.797957 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.798533 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.799549 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.799785 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.799859 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:10.802157 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:10.802158 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:10.802159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:10.802167 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:10.803399 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:10.803851 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:10.803853 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:10.803853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:10.803919 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:10.804232 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:10.804233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:10.804234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:10.804370 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:10.804451 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:10.804727 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 08:26:10.804820 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:26:10.805109 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:10.796025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:40903)
0619 08:26:10.796205 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:10.796248 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:10.796784 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:10.797050 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:10.797054 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:10.797630 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:10.797849 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:10.797855 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.798634 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.798635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:10.799449 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:10.799451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.799880 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.799881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:10.800004 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:10.800616 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:10.800625 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:10.802058 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:10.803486 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:10.803494 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:10.803496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:10.803756 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:10.803997 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:10.803997 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:10.803998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:10.804125 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:10.804440 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:10.804448 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:10.804590 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 08:26:10.804694 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 08:26:10.804873 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":85,"threads_started":1}
I20260619 08:26:10.805660 7859 heartbeater.cc:347] Connected to a master server at 127.25.127.190:34081
I20260619 08:26:10.805778 7859 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:10.805961 7859 heartbeater.cc:511] Master 127.25.127.190:34081 requested a full tablet report, sending...
I20260619 08:26:10.806327 7398 ts_manager.cc:194] Registered new tserver with Master: 7135752d05f347399a2580f52c9abe22 (127.25.127.131:39281)
I20260619 08:26:10.806877 7398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:40903
I20260619 08:26:10.807894 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:26:10.814742 7444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:10.809288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42966 (local address 127.25.127.190:34081)
0619 08:26:10.809360 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:10.809363 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:10.809412 (+ 49us) server_negotiation.cc:407] Connection header received
0619 08:26:10.809480 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:10.809481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:10.809489 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:10.809526 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:10.810061 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.810516 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.811241 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.811419 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.811468 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:10.812616 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:10.812617 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:10.812617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:10.812622 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:10.813759 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:10.814054 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:10.814055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:10.814055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:10.814102 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:10.814320 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:10.814320 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:10.814321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:10.814453 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:10.814521 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:10.814608 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 08:26:10.814660 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 08:26:10.816260 7398 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:26:10.817734 7398 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:26:10.824030 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:10.819726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:42971 (local address 127.0.0.1:57736)
0619 08:26:10.819899 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:10.819939 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:10.820005 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:10.820354 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:10.820358 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:10.820360 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:10.820460 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:10.820463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.821516 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.821517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:10.822754 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:10.822757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.823709 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.823710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:10.823805 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:10.823820 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 08:26:10.823889 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 08:26:10.823945 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":72,"threads_started":1}
I20260619 08:26:10.824131 7871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:10.819973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38070 (local address 127.25.127.131:39281)
0619 08:26:10.820373 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:10.820378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:10.820387 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:26:10.820421 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:10.820423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:10.820469 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:10.820546 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:10.820669 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.821776 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.823024 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.823755 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.823859 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:10.823912 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:10.823995 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 08:26:10.824041 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":72,"threads_started":1}
I20260619 08:26:10.824451 7870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:10.819907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53210 (local address 127.25.127.129:33441)
0619 08:26:10.820681 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:10.820686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:10.820696 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:26:10.820737 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:10.820739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:10.820766 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:10.820842 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:10.820952 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.821987 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.823161 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.823972 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.824038 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:10.824086 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:10.824314 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 08:26:10.824360 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":84,"threads_started":1}
I20260619 08:26:10.824571 7872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:10.819986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57736 (local address 127.25.127.130:42971)
0619 08:26:10.820178 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:10.820182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:10.820193 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:26:10.820233 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:10.820235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:10.820257 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:10.820326 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:10.820547 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.821416 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.822841 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.823620 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.823710 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:10.823781 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:10.824431 (+ 650us) server_negotiation.cc:299] Negotiation successful
0619 08:26:10.824484 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260619 08:26:10.824138 7868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:10.819728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:39281 (local address 127.0.0.1:38070)
0619 08:26:10.820106 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:10.820115 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:10.820152 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:10.820548 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:10.820549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:10.820551 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:10.820590 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:10.820591 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.821882 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.821882 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:10.822942 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:10.822944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.823859 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.823860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:10.823919 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:10.823921 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:26:10.823980 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 08:26:10.824012 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":37,"thread_start_us":69,"threads_started":1}
I20260619 08:26:10.824404 7869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:10.819730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:33441 (local address 127.0.0.1:53210)
0619 08:26:10.820196 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:10.820205 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:10.820244 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:10.820842 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:10.820842 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:10.820845 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:10.820885 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:10.820887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.822079 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.822080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:10.823074 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:10.823077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.824121 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.824122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:10.824213 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:10.824217 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:26:10.824283 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 08:26:10.824329 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":2,"thread_start_us":62,"threads_started":1}
I20260619 08:26:10.825384 7658 tablet_service.cc:1467] Processing CreateTablet for tablet acdf484d14224e1eae3e1dc02eb6764f (DEFAULT_TABLE table=test-table [id=ef3ee2a8159141e3a41742b9cf9d07d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:26:10.825485 7524 tablet_service.cc:1467] Processing CreateTablet for tablet acdf484d14224e1eae3e1dc02eb6764f (DEFAULT_TABLE table=test-table [id=ef3ee2a8159141e3a41742b9cf9d07d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:26:10.825699 7658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acdf484d14224e1eae3e1dc02eb6764f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:10.825876 7524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acdf484d14224e1eae3e1dc02eb6764f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:10.828262 7873 tablet_bootstrap.cc:492] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: Bootstrap starting.
I20260619 08:26:10.828536 7874 tablet_bootstrap.cc:492] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3: Bootstrap starting.
I20260619 08:26:10.828761 7793 tablet_service.cc:1467] Processing CreateTablet for tablet acdf484d14224e1eae3e1dc02eb6764f (DEFAULT_TABLE table=test-table [id=ef3ee2a8159141e3a41742b9cf9d07d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:26:10.829044 7873 tablet_bootstrap.cc:654] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:10.829072 7793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acdf484d14224e1eae3e1dc02eb6764f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:10.829134 7874 tablet_bootstrap.cc:654] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:10.829722 7874 log.cc:824] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:10.829722 7873 log.cc:824] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:10.830507 7873 tablet_bootstrap.cc:492] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: No bootstrap required, opened a new log
I20260619 08:26:10.830580 7873 ts_tablet_manager.cc:1397] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:26:10.830747 7874 tablet_bootstrap.cc:492] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3: No bootstrap required, opened a new log
I20260619 08:26:10.830813 7874 ts_tablet_manager.cc:1397] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:26:10.831902 7877 tablet_bootstrap.cc:492] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22: Bootstrap starting.
I20260619 08:26:10.832549 7877 tablet_bootstrap.cc:654] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:10.832767 7873 raft_consensus.cc:348] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } }
I20260619 08:26:10.832799 7874 raft_consensus.cc:348] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } }
I20260619 08:26:10.832950 7874 raft_consensus.cc:374] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:10.832950 7873 raft_consensus.cc:374] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:10.832988 7874 raft_consensus.cc:729] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800160590984b53bfe805a885538ec3, State: Initialized, Role: FOLLOWER
I20260619 08:26:10.832988 7873 raft_consensus.cc:729] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade1c097ed4646dfa7d5cb6415591387, State: Initialized, Role: FOLLOWER
I20260619 08:26:10.833133 7877 log.cc:824] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:10.833133 7873 consensus_queue.cc:260] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } }
I20260619 08:26:10.833133 7874 consensus_queue.cc:260] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } }
I20260619 08:26:10.833412 7874 ts_tablet_manager.cc:1428] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 08:26:10.833412 7873 ts_tablet_manager.cc:1428] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:26:10.833556 7590 heartbeater.cc:503] Master 127.25.127.190:34081 was elected leader, sending a full tablet report...
I20260619 08:26:10.833828 7724 heartbeater.cc:503] Master 127.25.127.190:34081 was elected leader, sending a full tablet report...
I20260619 08:26:10.834995 7877 tablet_bootstrap.cc:492] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22: No bootstrap required, opened a new log
I20260619 08:26:10.835130 7877 ts_tablet_manager.cc:1397] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 08:26:10.836578 7877 raft_consensus.cc:348] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } }
I20260619 08:26:10.836706 7877 raft_consensus.cc:374] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:10.836730 7877 raft_consensus.cc:729] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7135752d05f347399a2580f52c9abe22, State: Initialized, Role: FOLLOWER
I20260619 08:26:10.836876 7877 consensus_queue.cc:260] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } }
I20260619 08:26:10.837164 7877 ts_tablet_manager.cc:1428] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:26:10.837211 7859 heartbeater.cc:503] Master 127.25.127.190:34081 was elected leader, sending a full tablet report...
W20260619 08:26:10.895337 7725 tablet.cc:2369] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:26:10.981370 7591 tablet.cc:2369] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:26:10.997184 7878 raft_consensus.cc:482] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:26:10.997355 7878 raft_consensus.cc:504] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } }
I20260619 08:26:10.997860 7878 leader_election.cc:288] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800160590984b53bfe805a885538ec3 (127.25.127.129:33441), 7135752d05f347399a2580f52c9abe22 (127.25.127.131:39281)
I20260619 08:26:11.001646 7726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:10.998087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:33441 (local address 127.25.127.130:48809)
0619 08:26:10.998182 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:10.998194 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:10.998244 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:10.998514 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:10.998516 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:10.998518 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:10.998598 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:10.998600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.999326 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.999327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.000438 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:11.000441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.001341 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.001342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.001435 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.001439 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:26:11.001510 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 08:26:11.001555 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260619 08:26:11.001636 7870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:10.998312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:48809 (local address 127.25.127.129:33441)
0619 08:26:10.998358 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:10.998360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:10.998369 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:26:10.998403 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:10.998404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:10.998416 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:10.998497 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:10.998700 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.999221 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.000533 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.001243 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.001332 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.001343 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:11.001519 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 08:26:11.001567 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:26:11.001780 7871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:10.998185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:34847 (local address 127.25.127.131:39281)
0619 08:26:10.998249 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:10.998251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:10.998463 (+ 212us) server_negotiation.cc:407] Connection header received
0619 08:26:10.998558 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:10.998560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:10.998572 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:10.998602 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:10.998812 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.999680 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.000873 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.001269 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.001359 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.001367 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:11.001602 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 08:26:11.001668 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 08:26:11.002036 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:10.998087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:39281 (local address 127.25.127.130:34847)
0619 08:26:10.998396 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:10.998404 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:10.998472 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:10.998669 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:10.998669 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:10.998671 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:10.998733 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:10.998734 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:10.999792 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:10.999793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.000782 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:11.000785 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.001404 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.001405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.001502 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.001508 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 08:26:11.001869 (+ 361us) client_negotiation.cc:240] Negotiation successful
0619 08:26:11.001931 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":141,"thread_start_us":96,"threads_started":1}
I20260619 08:26:11.002058 7544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acdf484d14224e1eae3e1dc02eb6764f" candidate_uuid: "ade1c097ed4646dfa7d5cb6415591387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800160590984b53bfe805a885538ec3" is_pre_election: true
I20260619 08:26:11.002216 7544 raft_consensus.cc:2454] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ade1c097ed4646dfa7d5cb6415591387 in term 0.
I20260619 08:26:11.002463 7613 leader_election.cc:302] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a800160590984b53bfe805a885538ec3, ade1c097ed4646dfa7d5cb6415591387; no voters:
I20260619 08:26:11.002490 7813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acdf484d14224e1eae3e1dc02eb6764f" candidate_uuid: "ade1c097ed4646dfa7d5cb6415591387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7135752d05f347399a2580f52c9abe22" is_pre_election: true
I20260619 08:26:11.002626 7878 raft_consensus.cc:2781] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:26:11.002630 7813 raft_consensus.cc:2454] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ade1c097ed4646dfa7d5cb6415591387 in term 0.
I20260619 08:26:11.002707 7878 raft_consensus.cc:482] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:26:11.002770 7878 raft_consensus.cc:3037] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:11.003500 7878 raft_consensus.cc:504] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } }
I20260619 08:26:11.003731 7878 leader_election.cc:288] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [CANDIDATE]: Term 1 election: Requested vote from peers a800160590984b53bfe805a885538ec3 (127.25.127.129:33441), 7135752d05f347399a2580f52c9abe22 (127.25.127.131:39281)
I20260619 08:26:11.003825 7813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acdf484d14224e1eae3e1dc02eb6764f" candidate_uuid: "ade1c097ed4646dfa7d5cb6415591387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7135752d05f347399a2580f52c9abe22"
I20260619 08:26:11.003901 7544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acdf484d14224e1eae3e1dc02eb6764f" candidate_uuid: "ade1c097ed4646dfa7d5cb6415591387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800160590984b53bfe805a885538ec3"
I20260619 08:26:11.003917 7813 raft_consensus.cc:3037] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:11.003973 7544 raft_consensus.cc:3037] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:11.004879 7813 raft_consensus.cc:2454] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ade1c097ed4646dfa7d5cb6415591387 in term 1.
I20260619 08:26:11.004879 7544 raft_consensus.cc:2454] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ade1c097ed4646dfa7d5cb6415591387 in term 1.
I20260619 08:26:11.005076 7614 leader_election.cc:302] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7135752d05f347399a2580f52c9abe22, ade1c097ed4646dfa7d5cb6415591387; no voters:
I20260619 08:26:11.005173 7878 raft_consensus.cc:2781] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:26:11.005318 7878 raft_consensus.cc:686] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [term 1 LEADER]: Becoming Leader. State: Replica: ade1c097ed4646dfa7d5cb6415591387, State: Running, Role: LEADER
I20260619 08:26:11.005415 7878 consensus_queue.cc:237] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } }
I20260619 08:26:11.006321 7398 catalog_manager.cc:5495] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 reported cstate change: term changed from 0 to 1, leader changed from <none> to ade1c097ed4646dfa7d5cb6415591387 (127.25.127.130). New cstate: current_term: 1 leader_uuid: "ade1c097ed4646dfa7d5cb6415591387" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } health_report { overall_health: UNKNOWN } } }
I20260619 08:26:11.012256 7872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.009451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57746 (local address 127.25.127.130:42971)
0619 08:26:11.009495 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:11.009497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:11.009530 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:26:11.009617 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:11.009618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:11.009671 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:11.009752 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:26:11.009899 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.010365 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.011122 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.011355 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.011431 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.011688 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:26:11.011829 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:26:11.011926 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:11.012105 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 08:26:11.012126 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:26:11.014657 7813 raft_consensus.cc:1261] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22 [term 1 FOLLOWER]: Refusing update from remote peer ade1c097ed4646dfa7d5cb6415591387: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:26:11.014652 7544 raft_consensus.cc:1261] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3 [term 1 FOLLOWER]: Refusing update from remote peer ade1c097ed4646dfa7d5cb6415591387: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:26:11.015024 7883 consensus_queue.cc:1035] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:26:11.015184 7878 consensus_queue.cc:1035] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:26:11.017897 7886 mvcc.cc:205] Tried to move back new op lower bound from 7298488610873245696 to 7298488610838908928. Current Snapshot: MvccSnapshot[applied={T|T < 7298488610873245696}]
I20260619 08:26:11.018402 7887 mvcc.cc:205] Tried to move back new op lower bound from 7298488610873245696 to 7298488610838908928. Current Snapshot: MvccSnapshot[applied={T|T < 7298488610873245696}]
I20260619 08:26:11.019241 7888 mvcc.cc:205] Tried to move back new op lower bound from 7298488610873245696 to 7298488610838908928. Current Snapshot: MvccSnapshot[applied={T|T < 7298488610873245696 or (T in {7298488610873245696})}]
I20260619 08:26:11.026666 7397 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:26:11.026788 7397 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:26:11.027642 7397 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8: Sending DeleteTablet for 3 replicas of tablet acdf484d14224e1eae3e1dc02eb6764f
I20260619 08:26:11.028017 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 7460
I20260619 08:26:11.028126 7524 tablet_service.cc:1514] Processing DeleteTablet for tablet acdf484d14224e1eae3e1dc02eb6764f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:26:11 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:53210
I20260619 08:26:11.030608 7793 tablet_service.cc:1514] Processing DeleteTablet for tablet acdf484d14224e1eae3e1dc02eb6764f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:26:11 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:38070
I20260619 08:26:11.030639 7658 tablet_service.cc:1514] Processing DeleteTablet for tablet acdf484d14224e1eae3e1dc02eb6764f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:26:11 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:57736
I20260619 08:26:11.032338 7896 tablet_replica.cc:330] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: stopping tablet replica
I20260619 08:26:11.032529 7896 raft_consensus.cc:2229] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:26:11.032769 7896 raft_consensus.cc:2258] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:26:11.033022 7897 tablet_replica.cc:330] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22: stopping tablet replica
I20260619 08:26:11.033226 7897 raft_consensus.cc:2229] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:26:11.033424 7897 raft_consensus.cc:2258] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:26:11.034015 7897 ts_tablet_manager.cc:1905] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 08:26:11.035413 7384 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.127.129:33441 (error 108)
I20260619 08:26:11.035663 7896 ts_tablet_manager.cc:1905] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:26:11.035701 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 7595
I20260619 08:26:11.037153 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.036987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:33441 (local address 127.0.0.1:53226)
0619 08:26:11.037049 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:11.037102 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.129:33441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 08:26:11.037213 7897 ts_tablet_manager.cc:1918] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:26:11.037285 7897 log.cc:1196] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/acdf484d14224e1eae3e1dc02eb6764f
I20260619 08:26:11.037631 7897 ts_tablet_manager.cc:1939] T acdf484d14224e1eae3e1dc02eb6764f P 7135752d05f347399a2580f52c9abe22: Deleting consensus metadata
W20260619 08:26:11.037894 7384 catalog_manager.cc:4584] TS a800160590984b53bfe805a885538ec3 (127.25.127.129:33441): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet acdf484d14224e1eae3e1dc02eb6764f: Network error: Client connection negotiation failed: client connection to 127.25.127.129:33441: connect: Connection refused (error 111)
I20260619 08:26:11.038113 7385 catalog_manager.cc:4841] TS 7135752d05f347399a2580f52c9abe22 (127.25.127.131:39281): tablet acdf484d14224e1eae3e1dc02eb6764f (table test-table [id=ef3ee2a8159141e3a41742b9cf9d07d3]) successfully deleted
I20260619 08:26:11.041069 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 7729
I20260619 08:26:11.041258 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.041123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:42971 (local address 127.0.0.1:57756)
0619 08:26:11.041182 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:11.041210 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.130:42971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 08:26:11.046627 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 7366
I20260619 08:26:11.052551 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:34081
--webserver_interface=127.25.127.190
--webserver_port=37101
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:34081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:26:11.133193 7899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:11.133417 7899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:11.133441 7899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:11.134650 7899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:26:11.134698 7899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:11.134712 7899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:11.134725 7899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:26:11.134790 7899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:26:11.136401 7899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:34081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:34081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=37101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:11.136672 7899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:11.136937 7899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:11.146839 7908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:11.146870 7906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:11.146879 7899 server_base.cc:1034] running on GCE node
W20260619 08:26:11.146840 7905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:11.147316 7899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:11.147650 7899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:11.148818 7899 hybrid_clock.cc:648] HybridClock initialized: now 1781857571148802 us; error 26 us; skew 500 ppm
Jun 19 08:26:11 dist-test-slave-5rhg krb5kdc[7353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857571, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:11.151432 7899 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:26:11.151975 7899 webserver.cc:460] Webserver started at http://127.25.127.190:37101/ using document root <none> and password file <none>
I20260619 08:26:11.152285 7899 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:11.152367 7899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:11.153842 7899 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:26:11.154526 7916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:11.154707 7899 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:26:11.154831 7899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "883181651454435081075e99cad82cb8"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "6583794567092c68f15437a81984576f"
server_key_iv: "27c93fdb860ee77ec60a3905c318c9fe"
server_key_version: "encryptionkey@0"
I20260619 08:26:11.155257 7899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:11.172554 7899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:11.174533 7899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:11.174690 7899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:11.179111 7899 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:34081
I20260619 08:26:11.179143 7969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:34081 every 8 connection(s)
I20260619 08:26:11.179560 7899 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:26:11.181316 7970 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:26:11.181990 7970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8: Bootstrap starting.
I20260619 08:26:11.187652 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 7899
I20260619 08:26:11.187984 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:33441
--local_ip_for_outbound_sockets=127.25.127.129
--tserver_master_addrs=127.25.127.190:34081
--webserver_port=33731
--webserver_interface=127.25.127.129
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:26:11.191035 7970 log.cc:824] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:11.193296 7970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:26:11.193602 7970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8: Bootstrap complete.
I20260619 08:26:11.195981 7970 raft_consensus.cc:348] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883181651454435081075e99cad82cb8" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 34081 } }
I20260619 08:26:11.196310 7970 raft_consensus.cc:729] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 883181651454435081075e99cad82cb8, State: Initialized, Role: FOLLOWER
I20260619 08:26:11.196509 7970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883181651454435081075e99cad82cb8" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 34081 } }
I20260619 08:26:11.196589 7970 raft_consensus.cc:388] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:26:11.196652 7970 raft_consensus.cc:482] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:26:11.196705 7970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:26:11.197772 7970 raft_consensus.cc:504] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883181651454435081075e99cad82cb8" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 34081 } }
I20260619 08:26:11.197899 7970 leader_election.cc:302] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 883181651454435081075e99cad82cb8; no voters:
I20260619 08:26:11.198117 7970 leader_election.cc:288] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 08:26:11.198160 7974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 08:26:11.198320 7974 raft_consensus.cc:686] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 2 LEADER]: Becoming Leader. State: Replica: 883181651454435081075e99cad82cb8, State: Running, Role: LEADER
I20260619 08:26:11.198419 7970 sys_catalog.cc:564] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:26:11.198472 7974 consensus_queue.cc:237] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883181651454435081075e99cad82cb8" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 34081 } }
I20260619 08:26:11.198748 7974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 883181651454435081075e99cad82cb8. Latest consensus state: current_term: 2 leader_uuid: "883181651454435081075e99cad82cb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883181651454435081075e99cad82cb8" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 34081 } } }
I20260619 08:26:11.198829 7974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [sys.catalog]: This master's current role is: LEADER
I20260619 08:26:11.198907 7975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "883181651454435081075e99cad82cb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883181651454435081075e99cad82cb8" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 34081 } } }
I20260619 08:26:11.199044 7975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [sys.catalog]: This master's current role is: LEADER
I20260619 08:26:11.199040 7979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:26:11.200553 7979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:26:11.200671 7979 catalog_manager.cc:1259] Loaded cluster ID: ebf317f7bb724fcabc1edc6c8b60bcb1
I20260619 08:26:11.200763 7979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:26:11.201246 7979 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 08:26:11.203924 7979 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 08:26:11.205938 7979 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:26:11.206147 7979 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8: Loaded TSK: 0
I20260619 08:26:11.206390 7979 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 08:26:11.290647 7972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:11.290920 7972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:11.290967 7972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:11.292331 7972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:11.292402 7972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:11.292451 7972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:26:11.294101 7972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:33441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=33731
--tserver_master_addrs=127.25.127.190:34081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:11.294389 7972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:11.294638 7972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:11.300994 7997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:11.301040 7998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:11.301149 7972 server_base.cc:1034] running on GCE node
W20260619 08:26:11.301038 8000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:11.301481 7972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:11.301950 7972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:11.303143 7972 hybrid_clock.cc:648] HybridClock initialized: now 1781857571303118 us; error 41 us; skew 500 ppm
Jun 19 08:26:11 dist-test-slave-5rhg krb5kdc[7353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857571, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:11.305903 7972 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:26:11.306463 7972 webserver.cc:460] Webserver started at http://127.25.127.129:33731/ using document root <none> and password file <none>
I20260619 08:26:11.306713 7972 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:11.306798 7972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:11.308280 7972 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:26:11.309087 8007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:11.309269 7972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:11.309360 7972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "a800160590984b53bfe805a885538ec3"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "b8b201a0c74d7dd6dff6553889bfe231"
server_key_iv: "1447816804169109e55ad58b7c1dac2e"
server_key_version: "encryptionkey@0"
I20260619 08:26:11.309754 7972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:11.322801 7972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:11.325273 7972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:11.325443 7972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:11.325714 7972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:11.326227 8014 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:26:11.327673 7972 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:26:11.327729 7972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:26:11.327757 7972 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:26:11.328531 7972 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:26:11.328576 7972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:11.328619 8014 tablet_bootstrap.cc:492] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3: Bootstrap starting.
I20260619 08:26:11.336457 7972 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:33441
I20260619 08:26:11.336530 8122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:33441 every 8 connection(s)
I20260619 08:26:11.337131 7972 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:26:11.341639 8014 log.cc:824] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:11.343713 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 7972
I20260619 08:26:11.344055 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:42971
--local_ip_for_outbound_sockets=127.25.127.130
--tserver_master_addrs=127.25.127.190:34081
--webserver_port=40077
--webserver_interface=127.25.127.130
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:26:11.344650 8014 tablet_bootstrap.cc:492] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:26:11.345000 8014 tablet_bootstrap.cc:492] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3: Bootstrap complete.
I20260619 08:26:11.346158 8014 ts_tablet_manager.cc:1397] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.013s
Jun 19 08:26:11 dist-test-slave-5rhg krb5kdc[7353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857571, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:11.347353 8014 raft_consensus.cc:348] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } }
I20260619 08:26:11.348515 8014 raft_consensus.cc:729] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800160590984b53bfe805a885538ec3, State: Initialized, Role: FOLLOWER
I20260619 08:26:11.348729 8014 consensus_queue.cc:260] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } }
I20260619 08:26:11.349045 8014 ts_tablet_manager.cc:1428] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260619 08:26:11.351351 8125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.337866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:54909)
0619 08:26:11.338074 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:11.338121 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:11.338721 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:11.339302 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:11.339306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:11.339664 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:11.339928 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:11.339934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.341761 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.341762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.343015 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:11.343019 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.343916 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.343917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.344064 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.344705 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:11.344716 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:11.347072 (+ 2356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:11.349475 (+ 2403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:11.349480 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:11.349483 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:11.349792 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:11.350109 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:11.350110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:11.350111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:11.350233 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:11.350712 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:11.350717 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:11.350870 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 08:26:11.350974 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 08:26:11.351142 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":115,"threads_started":1}
I20260619 08:26:11.351861 8126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.338010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:54909 (local address 127.25.127.190:34081)
0619 08:26:11.338239 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:11.338248 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:11.338286 (+ 38us) server_negotiation.cc:407] Connection header received
0619 08:26:11.338995 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:11.339006 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:11.339083 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:11.339266 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:11.340079 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.341633 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.343103 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.343688 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.343801 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.347212 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:11.347217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:11.347218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:11.347232 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:11.349394 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:11.349899 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:11.349902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:11.349903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:11.350032 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:11.350307 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:11.350308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:11.350308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:11.350632 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:11.350734 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:11.351518 (+ 784us) server_negotiation.cc:299] Negotiation successful
0619 08:26:11.351653 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":95,"threads_started":1}
I20260619 08:26:11.352572 8123 heartbeater.cc:347] Connected to a master server at 127.25.127.190:34081
I20260619 08:26:11.352695 8123 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:11.352911 8123 heartbeater.cc:511] Master 127.25.127.190:34081 requested a full tablet report, sending...
I20260619 08:26:11.353790 7934 ts_manager.cc:194] Registered new tserver with Master: a800160590984b53bfe805a885538ec3 (127.25.127.129:33441)
I20260619 08:26:11.354875 7934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:54909
I20260619 08:26:11.356006 8123 heartbeater.cc:503] Master 127.25.127.190:34081 was elected leader, sending a full tablet report...
I20260619 08:26:11.360780 8131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.356185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53240 (local address 127.25.127.129:33441)
0619 08:26:11.356371 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:11.356376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:11.356732 (+ 356us) server_negotiation.cc:407] Connection header received
0619 08:26:11.356992 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:11.356995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:11.357011 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:11.357110 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:11.357309 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.358098 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.359833 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.360413 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.360479 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.360516 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:11.360692 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 08:26:11.360725 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
I20260619 08:26:11.360821 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.355280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:33441 (local address 127.0.0.1:53240)
0619 08:26:11.356532 (+ 1252us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:11.356578 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:11.356848 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:11.357127 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:11.357130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:11.357133 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:11.357220 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:11.357223 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.358177 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.358178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.359655 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:11.359659 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.360492 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.360493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.360582 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.360597 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 08:26:11.360673 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 08:26:11.360737 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1218,"thread_start_us":89,"threads_started":1}
I20260619 08:26:11.361395 8056 tablet_service.cc:1514] Processing DeleteTablet for tablet acdf484d14224e1eae3e1dc02eb6764f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:26:11 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:53240
I20260619 08:26:11.361917 8132 tablet_replica.cc:330] stopping tablet replica
I20260619 08:26:11.362026 8132 raft_consensus.cc:2229] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:26:11.362094 8132 raft_consensus.cc:2258] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:26:11.362406 8132 ts_tablet_manager.cc:1905] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:26:11.364722 8132 ts_tablet_manager.cc:1918] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:26:11.364801 8132 log.cc:1196] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/wal/wals/acdf484d14224e1eae3e1dc02eb6764f
I20260619 08:26:11.365027 8132 ts_tablet_manager.cc:1939] T acdf484d14224e1eae3e1dc02eb6764f P a800160590984b53bfe805a885538ec3: Deleting consensus metadata
I20260619 08:26:11.365553 7919 catalog_manager.cc:4841] TS a800160590984b53bfe805a885538ec3 (127.25.127.129:33441): tablet acdf484d14224e1eae3e1dc02eb6764f (table test-table [id=ef3ee2a8159141e3a41742b9cf9d07d3]) successfully deleted
W20260619 08:26:11.436353 8128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:11.436631 8128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:11.436674 8128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:11.437902 8128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:11.437989 8128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:11.438037 8128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:26:11.439793 8128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:42971
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=40077
--tserver_master_addrs=127.25.127.190:34081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:11.440114 8128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:11.440418 8128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:11.445804 8140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:11.445914 8128 server_base.cc:1034] running on GCE node
W20260619 08:26:11.445745 8137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:11.445822 8138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:11.446242 8128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:11.446542 8128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:11.447706 8128 hybrid_clock.cc:648] HybridClock initialized: now 1781857571447692 us; error 27 us; skew 500 ppm
Jun 19 08:26:11 dist-test-slave-5rhg krb5kdc[7353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857571, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:11.450472 8128 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:26:11.450977 8128 webserver.cc:460] Webserver started at http://127.25.127.130:40077/ using document root <none> and password file <none>
I20260619 08:26:11.451236 8128 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:11.451320 8128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:11.452814 8128 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:26:11.453539 8147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:11.453711 8128 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:26:11.453832 8128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "ade1c097ed4646dfa7d5cb6415591387"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "57291e2a88cb8ae433dd7309d1c98567"
server_key_iv: "217e891fb7a308d476fa47d8c98a04aa"
server_key_version: "encryptionkey@0"
I20260619 08:26:11.454193 8128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:11.470054 8128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:11.472028 8128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:11.472229 8128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:11.472513 8128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:11.472997 8154 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:26:11.474407 8128 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:26:11.474468 8128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:26:11.474514 8128 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:26:11.475169 8128 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:26:11.475226 8128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 08:26:11.475292 8154 tablet_bootstrap.cc:492] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: Bootstrap starting.
I20260619 08:26:11.477188 8154 log.cc:824] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:11.480489 8154 tablet_bootstrap.cc:492] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:26:11.480844 8154 tablet_bootstrap.cc:492] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: Bootstrap complete.
I20260619 08:26:11.481019 8154 ts_tablet_manager.cc:1397] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260619 08:26:11.481750 8128 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:42971
I20260619 08:26:11.481909 8263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:42971 every 8 connection(s)
I20260619 08:26:11.482151 8128 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:26:11.483126 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 8128
I20260619 08:26:11.482918 8154 raft_consensus.cc:348] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } }
I20260619 08:26:11.483297 8154 raft_consensus.cc:729] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade1c097ed4646dfa7d5cb6415591387, State: Initialized, Role: FOLLOWER
I20260619 08:26:11.483418 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:39281
--local_ip_for_outbound_sockets=127.25.127.131
--tserver_master_addrs=127.25.127.190:34081
--webserver_port=45767
--webserver_interface=127.25.127.131
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:26:11.483467 8154 consensus_queue.cc:260] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } }
I20260619 08:26:11.483764 8154 ts_tablet_manager.cc:1428] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
Jun 19 08:26:11 dist-test-slave-5rhg krb5kdc[7353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857571, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:11.492600 8126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.482975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:36987 (local address 127.25.127.190:34081)
0619 08:26:11.483024 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:11.483026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:11.483233 (+ 207us) server_negotiation.cc:407] Connection header received
0619 08:26:11.483420 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:11.483421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:11.483436 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:11.483496 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:11.484254 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.484860 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.485863 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.486226 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.486301 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.489540 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:11.489541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:11.489541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:11.489548 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:11.491013 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:11.491529 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:11.491530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:11.491530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:11.491583 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:11.491903 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:11.491903 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:11.491904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:11.492009 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:11.492060 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:11.492453 (+ 393us) server_negotiation.cc:299] Negotiation successful
0619 08:26:11.492530 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:26:11.492933 8266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.482860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:36987)
0619 08:26:11.483090 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:11.483136 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:11.483331 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:11.483555 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:11.483559 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:11.483917 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:11.484118 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:11.484123 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.484984 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.484985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.485772 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:11.485773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.486313 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.486314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.486419 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.487029 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:11.487040 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:11.489441 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:11.491101 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:11.491109 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:11.491111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:11.491453 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:11.491653 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:11.491654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:11.491654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:11.491822 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:11.492078 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:11.492086 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:11.492306 (+ 220us) client_negotiation.cc:769] Sending connection context
0619 08:26:11.492440 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 08:26:11.492639 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":101,"threads_started":1}
I20260619 08:26:11.493541 8264 heartbeater.cc:347] Connected to a master server at 127.25.127.190:34081
I20260619 08:26:11.493659 8264 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:11.493873 8264 heartbeater.cc:511] Master 127.25.127.190:34081 requested a full tablet report, sending...
I20260619 08:26:11.494513 7934 ts_manager.cc:194] Registered new tserver with Master: ade1c097ed4646dfa7d5cb6415591387 (127.25.127.130:42971)
I20260619 08:26:11.495122 7934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:36987
I20260619 08:26:11.496594 8264 heartbeater.cc:503] Master 127.25.127.190:34081 was elected leader, sending a full tablet report...
Jun 19 08:26:11 dist-test-slave-5rhg krb5kdc[7353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857571, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for kudu/127.25.127.130@KRBTEST.COM
I20260619 08:26:11.502985 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.494804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:42971 (local address 127.0.0.1:57758)
0619 08:26:11.494867 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:11.494878 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:11.494919 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:11.495185 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:11.495186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:11.495479 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:11.495516 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:11.495518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.496596 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.496597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.497316 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:11.497319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.497855 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.497856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.497935 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.498591 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:11.498597 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:11.499997 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:11.501767 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:11.501776 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:11.501778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:11.501990 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:11.502230 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:11.502231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:11.502231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:11.502306 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:11.502607 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:11.502614 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:11.502668 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 08:26:11.502758 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 08:26:11.502881 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 08:26:11.502996 8269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.494864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57758 (local address 127.25.127.130:42971)
0619 08:26:11.495050 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:11.495053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:11.495061 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:26:11.495091 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:11.495092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:11.495102 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:11.495199 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:11.495601 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.496512 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.497413 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.497772 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.497856 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.500090 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:11.500099 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:11.500100 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:11.500107 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:11.501677 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:11.502092 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:11.502096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:11.502096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:11.502161 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:11.502377 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:11.502378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:11.502378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:11.502533 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:11.502627 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:11.502794 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 08:26:11.502888 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":102,"threads_started":1}
I20260619 08:26:11.503393 8197 tablet_service.cc:1514] Processing DeleteTablet for tablet acdf484d14224e1eae3e1dc02eb6764f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:26:11 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:57758
I20260619 08:26:11.504004 8270 tablet_replica.cc:330] stopping tablet replica
I20260619 08:26:11.504122 8270 raft_consensus.cc:2229] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:26:11.504220 8270 raft_consensus.cc:2258] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:26:11.504540 8270 ts_tablet_manager.cc:1905] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:26:11.506997 8270 ts_tablet_manager.cc:1918] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:26:11.507076 8270 log.cc:1196] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/acdf484d14224e1eae3e1dc02eb6764f
I20260619 08:26:11.507300 8270 ts_tablet_manager.cc:1939] T acdf484d14224e1eae3e1dc02eb6764f P ade1c097ed4646dfa7d5cb6415591387: Deleting consensus metadata
I20260619 08:26:11.507831 7917 catalog_manager.cc:4841] TS ade1c097ed4646dfa7d5cb6415591387 (127.25.127.130:42971): tablet acdf484d14224e1eae3e1dc02eb6764f (table test-table [id=ef3ee2a8159141e3a41742b9cf9d07d3]) successfully deleted
W20260619 08:26:11.572305 8268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:11.572573 8268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:11.572614 8268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:11.573863 8268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:11.573936 8268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:11.573984 8268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:26:11.575445 8268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:39281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=45767
--tserver_master_addrs=127.25.127.190:34081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:11.575708 8268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:11.575937 8268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:11.581275 8276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:11.581183 8278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:11.581439 8268 server_base.cc:1034] running on GCE node
W20260619 08:26:11.581187 8275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:11.581738 8268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:11.582012 8268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:11.583186 8268 hybrid_clock.cc:648] HybridClock initialized: now 1781857571583175 us; error 39 us; skew 500 ppm
Jun 19 08:26:11 dist-test-slave-5rhg krb5kdc[7353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857571, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:11.585913 8268 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:26:11.586373 8268 webserver.cc:460] Webserver started at http://127.25.127.131:45767/ using document root <none> and password file <none>
I20260619 08:26:11.586602 8268 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:11.586683 8268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:11.588142 8268 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:11.589105 8285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:11.589305 8268 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:11.589414 8268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "7135752d05f347399a2580f52c9abe22"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "223b70c3a1a46eac0840669fb1efefac"
server_key_iv: "ea7d9e6c76ccb3fa7e01974049f3b0c0"
server_key_version: "encryptionkey@0"
I20260619 08:26:11.589843 8268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:11.606740 8268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:11.609524 8268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:11.609715 8268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:11.609954 8268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:11.610247 8268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:26:11.610282 8268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:11.610307 8268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:26:11.610321 8268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:11.616665 8268 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:39281
I20260619 08:26:11.616747 8398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:39281 every 8 connection(s)
I20260619 08:26:11.617098 8268 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:26:11.619341 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 8268
Jun 19 08:26:11 dist-test-slave-5rhg krb5kdc[7353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857571, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:11.627343 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.619800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42970 (local address 127.25.127.190:34081)
0619 08:26:11.619960 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:11.619962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:11.619971 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:26:11.619996 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:11.619997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:11.620008 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:11.620082 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:11.621361 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.622027 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.622799 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.623009 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.623081 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.624369 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:11.624371 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:11.624371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:11.624378 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:11.626094 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:11.626516 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:11.626517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:11.626518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:11.626574 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:11.626780 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:11.626781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:11.626781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:11.626884 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:11.627088 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:11.627124 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 08:26:11.627209 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":84,"threads_started":1}
I20260619 08:26:11.628428 8126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.617871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:37675 (local address 127.25.127.190:34081)
0619 08:26:11.617928 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:11.617930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:11.618183 (+ 253us) server_negotiation.cc:407] Connection header received
0619 08:26:11.618696 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:11.618698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:11.618712 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:11.618785 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:11.620097 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.620680 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.621940 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.622155 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.622284 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.625098 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:11.625099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:11.625100 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:11.625108 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:11.626864 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:11.627374 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:11.627375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:11.627375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:11.627447 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:11.627729 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:11.627730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:11.627730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:11.627876 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:11.627939 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:11.628298 (+ 359us) server_negotiation.cc:299] Negotiation successful
0619 08:26:11.628364 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:26:11.628711 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.617851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:37675)
0619 08:26:11.618034 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:11.618077 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:11.618575 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:11.618869 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:11.618873 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:11.619670 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:11.620001 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:11.620008 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.620782 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.620782 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.621816 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:11.621819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.622303 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.622304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.622462 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.623056 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:11.623065 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:11.624974 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:11.626941 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:11.626950 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:11.626952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:11.627183 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:11.627524 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:11.627525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:11.627526 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:11.627675 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:11.627953 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:11.627962 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:11.628134 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 08:26:11.628281 (+ 147us) client_negotiation.cc:240] Negotiation successful
0619 08:26:11.628433 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
I20260619 08:26:11.629247 8399 heartbeater.cc:347] Connected to a master server at 127.25.127.190:34081
I20260619 08:26:11.629344 8399 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:11.629527 8399 heartbeater.cc:511] Master 127.25.127.190:34081 requested a full tablet report, sending...
I20260619 08:26:11.629926 7934 ts_manager.cc:194] Registered new tserver with Master: 7135752d05f347399a2580f52c9abe22 (127.25.127.131:39281)
I20260619 08:26:11.630297 7934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:37675
I20260619 08:26:11.639241 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:26:11.642194 8126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.639644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42986 (local address 127.25.127.190:34081)
0619 08:26:11.639692 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:11.639694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:11.639814 (+ 120us) server_negotiation.cc:407] Connection header received
0619 08:26:11.639888 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:11.639888 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:11.639924 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:11.639986 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:26:11.640095 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.640636 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.641416 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.641597 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.641650 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.641808 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:26:11.641947 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:26:11.642036 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:11.642111 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 08:26:11.642136 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:26:11.642463 7934 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:42986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:26:11.644074 7934 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:26:11.646142 8056 tablet_service.cc:1467] Processing CreateTablet for tablet 8292fdcf793543ac9c19b4292c9cb8cc (DEFAULT_TABLE table=test-table [id=6973f9a9a061475c87a0da9fef44fe28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:26:11.646258 8197 tablet_service.cc:1467] Processing CreateTablet for tablet 8292fdcf793543ac9c19b4292c9cb8cc (DEFAULT_TABLE table=test-table [id=6973f9a9a061475c87a0da9fef44fe28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:26:11.646334 8056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8292fdcf793543ac9c19b4292c9cb8cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:11.646464 8197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8292fdcf793543ac9c19b4292c9cb8cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:11.647632 8014 tablet_bootstrap.cc:492] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3: Bootstrap starting.
I20260619 08:26:11.647773 8154 tablet_bootstrap.cc:492] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: Bootstrap starting.
I20260619 08:26:11.648010 8014 tablet_bootstrap.cc:654] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:11.648209 8154 tablet_bootstrap.cc:654] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:11.648612 8014 tablet_bootstrap.cc:492] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3: No bootstrap required, opened a new log
I20260619 08:26:11.648667 8014 ts_tablet_manager.cc:1397] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:11.648793 8014 raft_consensus.cc:348] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } }
I20260619 08:26:11.648847 8014 raft_consensus.cc:374] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:11.648860 8154 tablet_bootstrap.cc:492] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: No bootstrap required, opened a new log
I20260619 08:26:11.648880 8014 raft_consensus.cc:729] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800160590984b53bfe805a885538ec3, State: Initialized, Role: FOLLOWER
I20260619 08:26:11.648902 8154 ts_tablet_manager.cc:1397] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:11.648943 8014 consensus_queue.cc:260] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } }
I20260619 08:26:11.649039 8014 ts_tablet_manager.cc:1428] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:11.649163 8154 raft_consensus.cc:348] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } }
I20260619 08:26:11.649223 8154 raft_consensus.cc:374] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:11.649276 8154 raft_consensus.cc:729] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade1c097ed4646dfa7d5cb6415591387, State: Initialized, Role: FOLLOWER
I20260619 08:26:11.649320 8154 consensus_queue.cc:260] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } }
I20260619 08:26:11.649402 8154 ts_tablet_manager.cc:1428] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:26:11.650588 8267 raft_consensus.cc:482] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:26:11.650686 8267 raft_consensus.cc:504] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } }
I20260619 08:26:11.650995 8267 leader_election.cc:288] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7135752d05f347399a2580f52c9abe22 (127.25.127.131:39281), a800160590984b53bfe805a885538ec3 (127.25.127.129:33441)
I20260619 08:26:11.651932 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.645965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:39281 (local address 127.0.0.1:38084)
0619 08:26:11.646012 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:11.646023 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:11.646062 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:11.648355 (+ 2293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:11.648356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:11.648359 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:11.648434 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:11.648436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.649843 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.649844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.650712 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:11.650714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.651699 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.651700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.651774 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.651780 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 08:26:11.651811 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 08:26:11.651860 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 08:26:11.652355 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.647889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38084 (local address 127.25.127.131:39281)
0619 08:26:11.648107 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:11.648113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:11.648123 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:26:11.648194 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:11.648196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:11.648214 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:11.648519 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:11.648593 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.649723 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.650813 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.651608 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.652122 (+ 514us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.652222 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:11.652248 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 08:26:11.652297 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":114,"threads_started":1}
I20260619 08:26:11.653071 8333 tablet_service.cc:1467] Processing CreateTablet for tablet 8292fdcf793543ac9c19b4292c9cb8cc (DEFAULT_TABLE table=test-table [id=6973f9a9a061475c87a0da9fef44fe28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:26:11.653446 8333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8292fdcf793543ac9c19b4292c9cb8cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:11.654726 8131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.651184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:49293 (local address 127.25.127.129:33441)
0619 08:26:11.651404 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:11.651405 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:11.651501 (+ 96us) server_negotiation.cc:407] Connection header received
0619 08:26:11.651589 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:11.651590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:11.651599 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:11.651629 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:11.651852 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.652578 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.653675 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.654294 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.654367 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.654376 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:11.654620 (+ 244us) server_negotiation.cc:299] Negotiation successful
0619 08:26:11.654663 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197}
I20260619 08:26:11.654737 8408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.651140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:33441 (local address 127.25.127.130:49293)
0619 08:26:11.651452 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:11.651460 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:11.651521 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:11.651696 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:11.651697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:11.651698 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:11.651764 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:11.651765 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.652640 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.652641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.653577 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:11.653580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.654437 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.654438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.654506 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.654538 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 08:26:11.654611 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 08:26:11.654652 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":87,"threads_started":1}
I20260619 08:26:11.654846 8266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.651120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:39281 (local address 127.25.127.130:56791)
0619 08:26:11.651265 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:11.651275 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:11.651322 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:11.651529 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:11.651530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:11.651532 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:11.651590 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:11.651592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.652591 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.652592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.653691 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:11.653694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.654552 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.654553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:11.654632 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.654635 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:26:11.654692 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 08:26:11.654728 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":24}
I20260619 08:26:11.655123 8409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.651227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:56791 (local address 127.25.127.131:39281)
0619 08:26:11.651384 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:11.651386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:11.651394 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:26:11.651416 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:11.651417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:11.651440 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:11.651523 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:11.651676 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.652485 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.653775 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.654369 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.654437 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.654443 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:11.654997 (+ 554us) server_negotiation.cc:299] Negotiation successful
0619 08:26:11.655034 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
I20260619 08:26:11.655215 8077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8292fdcf793543ac9c19b4292c9cb8cc" candidate_uuid: "ade1c097ed4646dfa7d5cb6415591387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800160590984b53bfe805a885538ec3" is_pre_election: true
I20260619 08:26:11.655373 8077 raft_consensus.cc:2454] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ade1c097ed4646dfa7d5cb6415591387 in term 0.
I20260619 08:26:11.655601 8150 leader_election.cc:302] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a800160590984b53bfe805a885538ec3, ade1c097ed4646dfa7d5cb6415591387; no voters:
W20260619 08:26:11.655695 8151 leader_election.cc:341] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7135752d05f347399a2580f52c9abe22 (127.25.127.131:39281): Not found: Tablet not found: 8292fdcf793543ac9c19b4292c9cb8cc
I20260619 08:26:11.655720 8267 raft_consensus.cc:2781] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:26:11.655767 8267 raft_consensus.cc:482] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:26:11.655788 8267 raft_consensus.cc:3037] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:11.656562 8267 raft_consensus.cc:504] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } }
I20260619 08:26:11.656710 8267 leader_election.cc:288] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [CANDIDATE]: Term 1 election: Requested vote from peers 7135752d05f347399a2580f52c9abe22 (127.25.127.131:39281), a800160590984b53bfe805a885538ec3 (127.25.127.129:33441)
I20260619 08:26:11.656908 8077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8292fdcf793543ac9c19b4292c9cb8cc" candidate_uuid: "ade1c097ed4646dfa7d5cb6415591387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800160590984b53bfe805a885538ec3"
W20260619 08:26:11.656978 8151 leader_election.cc:341] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7135752d05f347399a2580f52c9abe22 (127.25.127.131:39281): Not found: Tablet not found: 8292fdcf793543ac9c19b4292c9cb8cc
I20260619 08:26:11.656997 8077 raft_consensus.cc:3037] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:11.657202 8410 tablet_bootstrap.cc:492] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22: Bootstrap starting.
I20260619 08:26:11.658280 8410 tablet_bootstrap.cc:654] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:11.658434 8077 raft_consensus.cc:2454] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ade1c097ed4646dfa7d5cb6415591387 in term 1.
I20260619 08:26:11.658603 8150 leader_election.cc:302] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a800160590984b53bfe805a885538ec3, ade1c097ed4646dfa7d5cb6415591387; no voters: 7135752d05f347399a2580f52c9abe22
I20260619 08:26:11.658617 8410 log.cc:824] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:11.658726 8267 raft_consensus.cc:2781] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:26:11.658896 8267 raft_consensus.cc:686] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [term 1 LEADER]: Becoming Leader. State: Replica: ade1c097ed4646dfa7d5cb6415591387, State: Running, Role: LEADER
I20260619 08:26:11.658988 8267 consensus_queue.cc:237] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } }
I20260619 08:26:11.659438 8410 tablet_bootstrap.cc:492] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22: No bootstrap required, opened a new log
I20260619 08:26:11.659516 8410 ts_tablet_manager.cc:1397] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:26:11.659679 7933 catalog_manager.cc:5495] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 reported cstate change: term changed from 0 to 1, leader changed from <none> to ade1c097ed4646dfa7d5cb6415591387 (127.25.127.130). New cstate: current_term: 1 leader_uuid: "ade1c097ed4646dfa7d5cb6415591387" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } health_report { overall_health: HEALTHY } } }
I20260619 08:26:11.661249 8410 raft_consensus.cc:348] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } }
I20260619 08:26:11.661422 8410 raft_consensus.cc:374] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:11.661470 8410 raft_consensus.cc:729] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7135752d05f347399a2580f52c9abe22, State: Initialized, Role: FOLLOWER
I20260619 08:26:11.661592 8410 consensus_queue.cc:260] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } }
I20260619 08:26:11.661849 8410 ts_tablet_manager.cc:1428] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 08:26:11.661931 8399 heartbeater.cc:503] Master 127.25.127.190:34081 was elected leader, sending a full tablet report...
I20260619 08:26:11.667362 8269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.664846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57766 (local address 127.25.127.130:42971)
0619 08:26:11.664905 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:11.664907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:11.664915 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:26:11.664947 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:11.664948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:11.664965 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:11.665056 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:26:11.665226 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.665754 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.666511 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:11.666749 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:11.666837 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:11.667016 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:26:11.667129 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:26:11.667221 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:11.667259 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 08:26:11.667280 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:26:11.668852 8353 raft_consensus.cc:3037] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:11.668915 8077 raft_consensus.cc:1261] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 1 FOLLOWER]: Refusing update from remote peer ade1c097ed4646dfa7d5cb6415591387: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:26:11.669276 8267 consensus_queue.cc:1035] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:26:11.669909 8353 raft_consensus.cc:1261] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22 [term 1 FOLLOWER]: Refusing update from remote peer ade1c097ed4646dfa7d5cb6415591387: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:26:11.670171 8267 consensus_queue.cc:1035] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:26:11.671649 8416 mvcc.cc:205] Tried to move back new op lower bound from 7298488613553418240 to 7298488613515919360. Current Snapshot: MvccSnapshot[applied={T|T < 7298488613553418240}]
I20260619 08:26:11.673390 8418 mvcc.cc:205] Tried to move back new op lower bound from 7298488613553418240 to 7298488613515919360. Current Snapshot: MvccSnapshot[applied={T|T < 7298488613553418240}]
I20260619 08:26:11.673837 7934 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:42986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:26:11.673943 7934 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:42986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:26:11.674598 7934 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8: Sending DeleteTablet for 3 replicas of tablet 8292fdcf793543ac9c19b4292c9cb8cc
I20260619 08:26:11.674890 8056 tablet_service.cc:1514] Processing DeleteTablet for tablet 8292fdcf793543ac9c19b4292c9cb8cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:26:11 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:53240
I20260619 08:26:11.674947 8197 tablet_service.cc:1514] Processing DeleteTablet for tablet 8292fdcf793543ac9c19b4292c9cb8cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:26:11 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:57758
I20260619 08:26:11.674983 8132 tablet_replica.cc:330] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3: stopping tablet replica
I20260619 08:26:11.674963 8333 tablet_service.cc:1514] Processing DeleteTablet for tablet 8292fdcf793543ac9c19b4292c9cb8cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:26:11 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:38084
I20260619 08:26:11.675029 8132 raft_consensus.cc:2229] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:26:11.675074 8132 raft_consensus.cc:2258] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:26:11.675149 8270 tablet_replica.cc:330] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: stopping tablet replica
I20260619 08:26:11.675189 8424 tablet_replica.cc:330] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22: stopping tablet replica
I20260619 08:26:11.675203 8270 raft_consensus.cc:2229] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:26:11.675277 8270 raft_consensus.cc:2258] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:26:11.675284 8424 raft_consensus.cc:2229] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:26:11.675346 8424 raft_consensus.cc:2258] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:26:11.675565 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 7972
I20260619 08:26:11.675735 8132 ts_tablet_manager.cc:1905] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:26:11.677227 8270 ts_tablet_manager.cc:1905] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:26:11.679061 8270 ts_tablet_manager.cc:1918] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:26:11.679111 8270 log.cc:1196] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/8292fdcf793543ac9c19b4292c9cb8cc
I20260619 08:26:11.679330 8270 ts_tablet_manager.cc:1939] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: Deleting consensus metadata
I20260619 08:26:11.679805 7917 catalog_manager.cc:4841] TS ade1c097ed4646dfa7d5cb6415591387 (127.25.127.130:42971): tablet 8292fdcf793543ac9c19b4292c9cb8cc (table test-table [id=6973f9a9a061475c87a0da9fef44fe28]) successfully deleted
W20260619 08:26:11.681850 7919 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.127.129:33441 (error 108)
I20260619 08:26:11.682176 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 8128
I20260619 08:26:11.683481 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.683342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:33441 (local address 127.0.0.1:53246)
0619 08:26:11.683400 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:11.683445 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.129:33441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 08:26:11.683604 7919 catalog_manager.cc:4584] TS a800160590984b53bfe805a885538ec3 (127.25.127.129:33441): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8292fdcf793543ac9c19b4292c9cb8cc: Network error: Client connection negotiation failed: client connection to 127.25.127.129:33441: connect: Connection refused (error 111)
I20260619 08:26:11.688094 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 8268
I20260619 08:26:11.693253 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 7899
I20260619 08:26:11.693297 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:11.693150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:39281 (local address 127.0.0.1:38094)
0619 08:26:11.693217 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:11.693251 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.131:39281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 08:26:11.698626 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:34081
--webserver_interface=127.25.127.190
--webserver_port=37101
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:34081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:26:11.780185 8427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:11.780406 8427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:11.780429 8427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:11.781617 8427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:26:11.781666 8427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:11.781682 8427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:11.781693 8427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:26:11.781749 8427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:26:11.783321 8427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:34081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:34081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=37101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:11.783589 8427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:11.783830 8427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:11.788645 8432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:11.788645 8435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:11.788684 8433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:11.788741 8427 server_base.cc:1034] running on GCE node
I20260619 08:26:11.789304 8427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:11.789614 8427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:11.790794 8427 hybrid_clock.cc:648] HybridClock initialized: now 1781857571790780 us; error 43 us; skew 500 ppm
Jun 19 08:26:11 dist-test-slave-5rhg krb5kdc[7353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857571, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:11.793310 8427 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:26:11.793802 8427 webserver.cc:460] Webserver started at http://127.25.127.190:37101/ using document root <none> and password file <none>
I20260619 08:26:11.794025 8427 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:11.794092 8427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:11.795387 8427 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:11.796074 8442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:11.796263 8427 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:11.796366 8427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "883181651454435081075e99cad82cb8"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "6583794567092c68f15437a81984576f"
server_key_iv: "27c93fdb860ee77ec60a3905c318c9fe"
server_key_version: "encryptionkey@0"
I20260619 08:26:11.796705 8427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:11.810567 8427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:11.812462 8427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:11.812610 8427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:11.816689 8427 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:34081
I20260619 08:26:11.816723 8494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:34081 every 8 connection(s)
I20260619 08:26:11.817106 8427 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:26:11.818667 8495 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:26:11.819250 8495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8: Bootstrap starting.
I20260619 08:26:11.823431 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 8427
I20260619 08:26:11.823725 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:33441
--local_ip_for_outbound_sockets=127.25.127.129
--tserver_master_addrs=127.25.127.190:34081
--webserver_port=33731
--webserver_interface=127.25.127.129
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:26:11.828336 8495 log.cc:824] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:11.830572 8495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:26:11.830909 8495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8: Bootstrap complete.
I20260619 08:26:11.833256 8495 raft_consensus.cc:348] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883181651454435081075e99cad82cb8" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 34081 } }
I20260619 08:26:11.833506 8495 raft_consensus.cc:729] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 883181651454435081075e99cad82cb8, State: Initialized, Role: FOLLOWER
I20260619 08:26:11.833710 8495 consensus_queue.cc:260] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883181651454435081075e99cad82cb8" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 34081 } }
I20260619 08:26:11.833818 8495 raft_consensus.cc:388] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:26:11.833873 8495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:26:11.833921 8495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 2 FOLLOWER]: Advancing to term 3
I20260619 08:26:11.834899 8495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883181651454435081075e99cad82cb8" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 34081 } }
I20260619 08:26:11.835021 8495 leader_election.cc:302] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 883181651454435081075e99cad82cb8; no voters:
I20260619 08:26:11.835220 8495 leader_election.cc:288] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 08:26:11.835265 8499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 08:26:11.835415 8499 raft_consensus.cc:686] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [term 3 LEADER]: Becoming Leader. State: Replica: 883181651454435081075e99cad82cb8, State: Running, Role: LEADER
I20260619 08:26:11.835522 8495 sys_catalog.cc:564] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:26:11.835567 8499 consensus_queue.cc:237] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883181651454435081075e99cad82cb8" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 34081 } }
I20260619 08:26:11.835842 8499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 883181651454435081075e99cad82cb8. Latest consensus state: current_term: 3 leader_uuid: "883181651454435081075e99cad82cb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883181651454435081075e99cad82cb8" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 34081 } } }
I20260619 08:26:11.835932 8499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [sys.catalog]: This master's current role is: LEADER
I20260619 08:26:11.835846 8500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "883181651454435081075e99cad82cb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "883181651454435081075e99cad82cb8" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 34081 } } }
I20260619 08:26:11.836115 8500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 883181651454435081075e99cad82cb8 [sys.catalog]: This master's current role is: LEADER
I20260619 08:26:11.836257 8506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:26:11.837116 8506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:26:11.837258 8506 catalog_manager.cc:1259] Loaded cluster ID: ebf317f7bb724fcabc1edc6c8b60bcb1
I20260619 08:26:11.837349 8506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 08:26:11.837798 8506 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fbb0690c9ad google::LogMessage::Fail() at ??:0
@ 0x7fbb0690f88a google::LogMessage::SendToLog() at ??:0
@ 0x7fbb0690c4de google::LogMessage::Flush() at ??:0
@ 0x7fbb0690da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fbb0e82d0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fbb0e8368b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fbb087da0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fbb087cf954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fbb096336db start_thread at ??:0
@ 0x7fbb0540a61f clone at ??:0
W20260619 08:26:11.918032 8497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:11.918223 8497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:11.918247 8497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:11.919387 8497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:11.919428 8497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:11.919454 8497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:26:11.920957 8497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:33441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=33731
--tserver_master_addrs=127.25.127.190:34081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:11.921213 8497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:11.921533 8497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:11.927800 8532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:11.927853 8531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:11.927999 8497 server_base.cc:1034] running on GCE node
W20260619 08:26:11.927968 8534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:11.928401 8497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:11.928678 8497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:11.929834 8497 hybrid_clock.cc:648] HybridClock initialized: now 1781857571929821 us; error 36 us; skew 500 ppm
Jun 19 08:26:11 dist-test-slave-5rhg krb5kdc[7353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857571, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:11.932512 8497 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:26:11.933027 8497 webserver.cc:460] Webserver started at http://127.25.127.129:33731/ using document root <none> and password file <none>
I20260619 08:26:11.933283 8497 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:11.933354 8497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:11.934693 8497 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:11.935459 8541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:11.935653 8497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:11.935760 8497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "a800160590984b53bfe805a885538ec3"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "b8b201a0c74d7dd6dff6553889bfe231"
server_key_iv: "1447816804169109e55ad58b7c1dac2e"
server_key_version: "encryptionkey@0"
I20260619 08:26:11.936272 8497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:11.954887 8497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:11.957459 8497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:11.957616 8497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:11.957896 8497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:11.958432 8548 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:26:11.959928 8497 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:26:11.959995 8497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:26:11.960032 8497 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:26:11.961006 8497 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:26:11.961056 8497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 08:26:11.961148 8548 tablet_bootstrap.cc:492] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3: Bootstrap starting.
I20260619 08:26:11.962909 8548 log.cc:824] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:11.965536 8548 tablet_bootstrap.cc:492] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:26:11.965842 8548 tablet_bootstrap.cc:492] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3: Bootstrap complete.
I20260619 08:26:11.966020 8548 ts_tablet_manager.cc:1397] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 08:26:11.967375 8548 raft_consensus.cc:348] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } }
I20260619 08:26:11.967635 8548 raft_consensus.cc:729] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800160590984b53bfe805a885538ec3, State: Initialized, Role: FOLLOWER
I20260619 08:26:11.967826 8548 consensus_queue.cc:260] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } }
I20260619 08:26:11.968163 8548 ts_tablet_manager.cc:1428] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260619 08:26:11.968611 8497 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:33441
I20260619 08:26:11.968690 8658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:33441 every 8 connection(s)
I20260619 08:26:11.969002 8497 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:26:11.978976 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 8497
I20260619 08:26:11.979300 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:42971
--local_ip_for_outbound_sockets=127.25.127.130
--tserver_master_addrs=127.25.127.190:34081
--webserver_port=40077
--webserver_interface=127.25.127.130
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:26:12.034507 8661 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:26:11.969737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:35937)
0619 08:26:11.969912 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:11.969945 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:11.970079 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:12.034277 (+ 64198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
W20260619 08:26:12.034783 8659 heartbeater.cc:644] Failed to heartbeat to 127.25.127.190:34081 (0 consecutive failures): Network error: Failed to ping master at 127.25.127.190:34081: Client connection negotiation failed: client connection to 127.25.127.190:34081: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 08:26:12.035148 8661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:12.035013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:39057)
0619 08:26:12.035074 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:12.035123 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 08:26:12.035532 8661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:12.035444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:55803)
0619 08:26:12.035493 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:12.035516 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 08:26:12.035665 8659 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 08:26:12.066128 8662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:12.066341 8662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:12.066367 8662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:12.067610 8662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:12.067656 8662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:12.067687 8662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:26:12.069249 8662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:42971
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=40077
--tserver_master_addrs=127.25.127.190:34081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:12.069501 8662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:12.069752 8662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:12.074834 8667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:12.074827 8670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:12.074827 8668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:12.075284 8662 server_base.cc:1034] running on GCE node
I20260619 08:26:12.075528 8662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:12.075798 8662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:12.076936 8662 hybrid_clock.cc:648] HybridClock initialized: now 1781857572076921 us; error 42 us; skew 500 ppm
Jun 19 08:26:12 dist-test-slave-5rhg krb5kdc[7353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857572, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:12.079468 8662 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:26:12.079924 8662 webserver.cc:460] Webserver started at http://127.25.127.130:40077/ using document root <none> and password file <none>
I20260619 08:26:12.080185 8662 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:12.080271 8662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:12.081604 8662 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:12.082186 8677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:12.082355 8662 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:26:12.082453 8662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "ade1c097ed4646dfa7d5cb6415591387"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "57291e2a88cb8ae433dd7309d1c98567"
server_key_iv: "217e891fb7a308d476fa47d8c98a04aa"
server_key_version: "encryptionkey@0"
I20260619 08:26:12.082842 8662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:12.095937 8662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:12.097921 8662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:12.098093 8662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:12.098341 8662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:12.098660 8662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:26:12.098724 8662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:12.098769 8662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:26:12.098800 8662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:12.104578 8662 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:42971
I20260619 08:26:12.104658 8790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:42971 every 8 connection(s)
I20260619 08:26:12.105036 8662 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:26:12.106065 8793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:12.105677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:43099)
0619 08:26:12.105833 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:12.105911 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
W20260619 08:26:12.106446 8791 heartbeater.cc:644] Failed to heartbeat to 127.25.127.190:34081 (0 consecutive failures): Network error: Failed to ping master at 127.25.127.190:34081: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
I20260619 08:26:12.106683 8793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:12.106617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:53477)
0619 08:26:12.106644 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:12.106667 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260619 08:26:12.106946 8793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:12.106895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:37433)
0619 08:26:12.106915 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:12.106934 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260619 08:26:12.107052 8791 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 08:26:12.115000 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 8662
I20260619 08:26:12.115269 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:39281
--local_ip_for_outbound_sockets=127.25.127.131
--tserver_master_addrs=127.25.127.190:34081
--webserver_port=45767
--webserver_interface=127.25.127.131
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:26:12.199993 8794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:12.200279 8794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:12.200309 8794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:12.201462 8794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:12.201499 8794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:12.201525 8794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:26:12.202955 8794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:39281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=45767
--tserver_master_addrs=127.25.127.190:34081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:12.203155 8794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:12.203347 8794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:12.208544 8803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:12.208690 8794 server_base.cc:1034] running on GCE node
W20260619 08:26:12.208544 8801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:12.208551 8800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:12.208981 8794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:12.209246 8794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:12.210389 8794 hybrid_clock.cc:648] HybridClock initialized: now 1781857572210379 us; error 28 us; skew 500 ppm
Jun 19 08:26:12 dist-test-slave-5rhg krb5kdc[7353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857572, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:12.212847 8794 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:26:12.213232 8794 webserver.cc:460] Webserver started at http://127.25.127.131:45767/ using document root <none> and password file <none>
I20260619 08:26:12.213482 8794 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:12.213555 8794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:12.214881 8794 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:26:12.215597 8810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:12.215806 8794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:26:12.215914 8794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "7135752d05f347399a2580f52c9abe22"
format_stamp: "Formatted at 2026-06-19 08:26:10 on dist-test-slave-5rhg"
server_key: "223b70c3a1a46eac0840669fb1efefac"
server_key_iv: "ea7d9e6c76ccb3fa7e01974049f3b0c0"
server_key_version: "encryptionkey@0"
I20260619 08:26:12.216322 8794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:12.226504 8794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:12.228543 8794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:12.228699 8794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:12.228924 8794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:12.229348 8817 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:26:12.230875 8794 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:26:12.230933 8794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:26:12.230994 8794 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:26:12.231652 8794 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:26:12.231706 8794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 08:26:12.231839 8817 tablet_bootstrap.cc:492] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22: Bootstrap starting.
I20260619 08:26:12.238619 8794 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:39281
I20260619 08:26:12.238674 8925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:39281 every 8 connection(s)
I20260619 08:26:12.239017 8794 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:26:12.240228 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:12.239706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:59711)
0619 08:26:12.239917 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:12.239997 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":130,"threads_started":1}
I20260619 08:26:12.240252 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 8794
W20260619 08:26:12.240588 8926 heartbeater.cc:644] Failed to heartbeat to 127.25.127.190:34081 (0 consecutive failures): Network error: Failed to ping master at 127.25.127.190:34081: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
I20260619 08:26:12.240892 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:12.240789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:52745)
0619 08:26:12.240832 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:12.240858 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 08:26:12.241289 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:12.241163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:46889)
0619 08:26:12.241211 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:12.241234 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 08:26:12.241438 8926 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 08:26:12.243407 8817 log.cc:824] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:12.245677 8817 tablet_bootstrap.cc:492] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:26:12.245971 8817 tablet_bootstrap.cc:492] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22: Bootstrap complete.
I20260619 08:26:12.246942 8817 ts_tablet_manager.cc:1397] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.009s
I20260619 08:26:12.248245 8817 raft_consensus.cc:348] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } }
I20260619 08:26:12.248528 8817 raft_consensus.cc:729] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7135752d05f347399a2580f52c9abe22, State: Initialized, Role: FOLLOWER
I20260619 08:26:12.248692 8817 consensus_queue.cc:260] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } }
I20260619 08:26:12.248935 8817 ts_tablet_manager.cc:1428] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260619 08:26:12.249255 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:12.249144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:48535)
0619 08:26:12.249201 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:12.249233 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 08:26:12.249516 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:12.249466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:47665)
0619 08:26:12.249488 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:12.249505 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
I20260619 08:26:13.036501 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:13.036241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:58417)
0619 08:26:13.036386 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:13.036438 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":61,"threads_started":1}
I20260619 08:26:13.107602 8933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:13.107396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:41885)
0619 08:26:13.107510 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:13.107551 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":55,"threads_started":1}
I20260619 08:26:13.250286 8934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:13.249970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:52015)
0619 08:26:13.250114 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:13.250163 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":63,"threads_started":1}
I20260619 08:26:13.738807 8935 raft_consensus.cc:482] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:26:13.738938 8935 raft_consensus.cc:504] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } }
I20260619 08:26:13.739364 8935 leader_election.cc:288] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7135752d05f347399a2580f52c9abe22 (127.25.127.131:39281), ade1c097ed4646dfa7d5cb6415591387 (127.25.127.130:42971)
Jun 19 08:26:13 dist-test-slave-5rhg krb5kdc[7353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857571, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.131@KRBTEST.COM
Jun 19 08:26:13 dist-test-slave-5rhg krb5kdc[7353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857571, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.130@KRBTEST.COM
I20260619 08:26:13.755223 8938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:13.739702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:32959 (local address 127.25.127.130:42971)
0619 08:26:13.739882 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:13.739895 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:13.739915 (+ 20us) server_negotiation.cc:407] Connection header received
0619 08:26:13.740530 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:13.740534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:13.740582 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:13.740725 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:13.741520 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:13.742448 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:13.743340 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:13.743776 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:13.743904 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:13.751412 (+ 7508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:13.751417 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:13.751417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:13.751433 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:13.753319 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:13.753898 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:13.753902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:13.753902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:13.754017 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:13.754280 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:13.754281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:13.754281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:13.754560 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:13.754660 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:13.754960 (+ 300us) server_negotiation.cc:299] Negotiation successful
0619 08:26:13.755096 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":57,"threads_started":1}
I20260619 08:26:13.755231 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:13.739710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:56225 (local address 127.25.127.131:39281)
0619 08:26:13.739875 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:13.739895 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:13.739915 (+ 20us) server_negotiation.cc:407] Connection header received
0619 08:26:13.740017 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:13.740019 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:13.740062 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:13.740233 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:13.741139 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:13.742225 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:13.743364 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:13.743810 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:13.743946 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:13.751099 (+ 7153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:13.751105 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:13.751105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:13.751118 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:13.753319 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:13.753922 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:13.753926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:13.753926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:13.754030 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:13.754300 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:13.754301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:13.754301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:13.754561 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:13.754659 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:13.754953 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 08:26:13.755121 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":56,"threads_started":1}
I20260619 08:26:13.755241 8936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:13.739579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:39281 (local address 127.25.127.129:56225)
0619 08:26:13.739718 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:13.739730 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:13.739791 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:13.740296 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:13.740299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:13.740784 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:13.741021 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:13.741027 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:13.742332 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:13.742333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:13.743208 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:13.743211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:13.743913 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:13.743914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:13.744011 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:13.749087 (+ 5076us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:13.749096 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:13.750935 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:13.753431 (+ 2496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:13.753440 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:13.753442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:13.753805 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:13.754130 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:13.754131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:13.754131 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:13.754214 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:13.754639 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:13.754646 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:13.754787 (+ 141us) client_negotiation.cc:769] Sending connection context
0619 08:26:13.754900 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 08:26:13.755108 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":10,"thread_start_us":59,"threads_started":1}
I20260619 08:26:13.755489 8937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:13.739590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:42971 (local address 127.25.127.129:32959)
0619 08:26:13.739747 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:13.739757 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:13.739822 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:13.740750 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:13.740751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:13.740970 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:13.741377 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:13.741380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:13.742540 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:13.742540 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:13.743228 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:13.743230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:13.743876 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:13.743878 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:13.744010 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:13.749084 (+ 5074us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:13.749096 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:13.751269 (+ 2173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:13.753449 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:13.753450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:13.753451 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:13.753798 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:13.754092 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:13.754092 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:13.754093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:13.754210 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:13.754639 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:13.754646 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:13.754787 (+ 141us) client_negotiation.cc:769] Sending connection context
0619 08:26:13.754900 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 08:26:13.755202 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":144,"thread_start_us":83,"threads_started":1}
I20260619 08:26:13.755798 8879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8292fdcf793543ac9c19b4292c9cb8cc" candidate_uuid: "a800160590984b53bfe805a885538ec3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7135752d05f347399a2580f52c9abe22" is_pre_election: true
I20260619 08:26:13.755967 8879 raft_consensus.cc:2454] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800160590984b53bfe805a885538ec3 in term 1.
I20260619 08:26:13.756314 8545 leader_election.cc:302] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7135752d05f347399a2580f52c9abe22, a800160590984b53bfe805a885538ec3; no voters:
I20260619 08:26:13.756458 8935 raft_consensus.cc:2781] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 08:26:13.756538 8935 raft_consensus.cc:482] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:26:13.756561 8935 raft_consensus.cc:3037] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 1 FOLLOWER]: Advancing to term 2
W20260619 08:26:13.756835 8542 leader_election.cc:341] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ade1c097ed4646dfa7d5cb6415591387 (127.25.127.130:42971): Not found: Tablet not found: 8292fdcf793543ac9c19b4292c9cb8cc
I20260619 08:26:13.758010 8935 raft_consensus.cc:504] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } }
I20260619 08:26:13.758170 8935 leader_election.cc:288] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [CANDIDATE]: Term 2 election: Requested vote from peers 7135752d05f347399a2580f52c9abe22 (127.25.127.131:39281), ade1c097ed4646dfa7d5cb6415591387 (127.25.127.130:42971)
I20260619 08:26:13.758389 8879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8292fdcf793543ac9c19b4292c9cb8cc" candidate_uuid: "a800160590984b53bfe805a885538ec3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7135752d05f347399a2580f52c9abe22"
W20260619 08:26:13.758494 8542 leader_election.cc:341] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer ade1c097ed4646dfa7d5cb6415591387 (127.25.127.130:42971): Not found: Tablet not found: 8292fdcf793543ac9c19b4292c9cb8cc
I20260619 08:26:13.758487 8879 raft_consensus.cc:3037] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22 [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:26:13.759579 8879 raft_consensus.cc:2454] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800160590984b53bfe805a885538ec3 in term 2.
I20260619 08:26:13.759765 8545 leader_election.cc:302] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7135752d05f347399a2580f52c9abe22, a800160590984b53bfe805a885538ec3; no voters: ade1c097ed4646dfa7d5cb6415591387
I20260619 08:26:13.759894 8935 raft_consensus.cc:2781] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 08:26:13.760041 8935 raft_consensus.cc:686] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [term 2 LEADER]: Becoming Leader. State: Replica: a800160590984b53bfe805a885538ec3, State: Running, Role: LEADER
I20260619 08:26:13.760151 8935 consensus_queue.cc:237] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } }
I20260619 08:26:13.760421 8937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:13.760317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:56691)
0619 08:26:13.760357 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:13.760392 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260619 08:26:14.108547 8942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:14.108214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:60111)
0619 08:26:14.108384 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:14.108447 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":77,"threads_started":1}
I20260619 08:26:14.251052 8943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:14.250822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:37649)
0619 08:26:14.250957 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:14.251000 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260619 08:26:14.304875 8879 raft_consensus.cc:1261] T 8292fdcf793543ac9c19b4292c9cb8cc P 7135752d05f347399a2580f52c9abe22 [term 2 FOLLOWER]: Refusing update from remote peer a800160590984b53bfe805a885538ec3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260619 08:26:14.305317 8944 consensus_queue.cc:1035] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 08:26:14.306432 8943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:14.306285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:57367)
0619 08:26:14.306346 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:14.306393 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 08:26:14.307492 8542 consensus_peers.cc:487] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3 -> Peer ade1c097ed4646dfa7d5cb6415591387 (127.25.127.130:42971): Couldn't send request to peer ade1c097ed4646dfa7d5cb6415591387. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8292fdcf793543ac9c19b4292c9cb8cc. This is attempt 1: this message will repeat every 5th retry.
I20260619 08:26:14.307615 8951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:14.307320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:53367)
0619 08:26:14.307541 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:14.307575 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":106,"threads_started":1}
I20260619 08:26:14.307600 8943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:14.307499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:54485)
0619 08:26:14.307541 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:14.307570 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 08:26:14.865890 8952 ts_tablet_manager.cc:927] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: Initiating tablet copy from peer a800160590984b53bfe805a885538ec3 (127.25.127.129:33441)
I20260619 08:26:14.866124 8952 tablet_copy_client.cc:316] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: tablet copy: Beginning tablet copy session from remote peer at address 127.25.127.129:33441
Jun 19 08:26:14 dist-test-slave-5rhg krb5kdc[7353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857572, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.129@KRBTEST.COM
I20260619 08:26:14.874715 8955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:14.866555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:56539 (local address 127.25.127.129:33441)
0619 08:26:14.866681 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:14.866687 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:14.866701 (+ 14us) server_negotiation.cc:407] Connection header received
0619 08:26:14.866788 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:14.866792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:14.866810 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:14.866899 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:14.867421 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:14.868129 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:14.869027 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:14.869349 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:14.869435 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:14.871651 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:14.871661 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:14.871662 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:14.871669 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:14.873423 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:14.873840 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:14.873843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:14.873844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:14.873905 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:14.874139 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:14.874140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:14.874140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:14.874293 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:14.874358 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:14.874569 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 08:26:14.874640 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":63,"threads_started":1}
I20260619 08:26:14.874732 8954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:14.866451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:33441 (local address 127.25.127.130:56539)
0619 08:26:14.866627 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:14.866648 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:14.866723 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:14.866905 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:14.866912 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:14.867248 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:14.867322 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:14.867325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:14.868279 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:14.868280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:14.868921 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:14.868923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:14.869432 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:14.869433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:14.869536 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:14.870030 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:14.870035 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:14.871549 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:14.873525 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:14.873533 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:14.873535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:14.873761 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:14.873997 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:14.873998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:14.873999 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:14.874070 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:14.874376 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:14.874385 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:14.874454 (+ 69us) client_negotiation.cc:769] Sending connection context
0619 08:26:14.874529 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 08:26:14.874659 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":115,"threads_started":1}
I20260619 08:26:14.875223 8632 tablet_copy_service.cc:140] P a800160590984b53bfe805a885538ec3: Received BeginTabletCopySession request for tablet 8292fdcf793543ac9c19b4292c9cb8cc from peer ade1c097ed4646dfa7d5cb6415591387 ({username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:56539)
I20260619 08:26:14.875299 8632 tablet_copy_service.cc:161] P a800160590984b53bfe805a885538ec3: Beginning new tablet copy session on tablet 8292fdcf793543ac9c19b4292c9cb8cc from peer ade1c097ed4646dfa7d5cb6415591387 at {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:56539: session id = ade1c097ed4646dfa7d5cb6415591387-8292fdcf793543ac9c19b4292c9cb8cc
I20260619 08:26:14.875816 8632 tablet_copy_source_session.cc:205] T 8292fdcf793543ac9c19b4292c9cb8cc P a800160590984b53bfe805a885538ec3: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 08:26:14.876880 8952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8292fdcf793543ac9c19b4292c9cb8cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:14.879535 8952 tablet_copy_client.cc:791] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: tablet copy: Starting download of 0 data blocks...
I20260619 08:26:14.879730 8952 tablet_copy_client.cc:655] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: tablet copy: Starting download of 1 WAL segments...
I20260619 08:26:14.880867 8952 tablet_copy_client.cc:531] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 08:26:14.881599 8952 tablet_bootstrap.cc:492] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: Bootstrap starting.
I20260619 08:26:14.883003 8952 log.cc:824] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:14.885772 8952 tablet_bootstrap.cc:492] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:26:14.886031 8952 tablet_bootstrap.cc:492] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: Bootstrap complete.
I20260619 08:26:14.886183 8952 ts_tablet_manager.cc:1397] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 08:26:14.887195 8952 raft_consensus.cc:348] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } }
I20260619 08:26:14.887377 8952 raft_consensus.cc:729] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade1c097ed4646dfa7d5cb6415591387, State: Initialized, Role: FOLLOWER
I20260619 08:26:14.887514 8952 consensus_queue.cc:260] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7135752d05f347399a2580f52c9abe22" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 39281 } } peers { permanent_uuid: "a800160590984b53bfe805a885538ec3" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33441 } } peers { permanent_uuid: "ade1c097ed4646dfa7d5cb6415591387" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 42971 } }
I20260619 08:26:14.887794 8952 ts_tablet_manager.cc:1428] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260619 08:26:14.888212 8954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:14.888027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:33307)
0619 08:26:14.888086 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:14.888132 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 08:26:14.888330 8632 tablet_copy_service.cc:342] P a800160590984b53bfe805a885538ec3: Request end of tablet copy session ade1c097ed4646dfa7d5cb6415591387-8292fdcf793543ac9c19b4292c9cb8cc received from {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:56539
I20260619 08:26:14.888396 8632 tablet_copy_service.cc:433] P a800160590984b53bfe805a885538ec3: ending tablet copy session ade1c097ed4646dfa7d5cb6415591387-8292fdcf793543ac9c19b4292c9cb8cc on tablet 8292fdcf793543ac9c19b4292c9cb8cc with peer ade1c097ed4646dfa7d5cb6415591387
I20260619 08:26:14.888579 8954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:14.888483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:59457)
0619 08:26:14.888521 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:14.888553 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 08:26:15.308313 8959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:15.308058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:32851)
0619 08:26:15.308223 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:15.308257 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":99,"threads_started":1}
I20260619 08:26:15.308333 8958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:15.308014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:41941)
0619 08:26:15.308241 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:15.308276 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":72,"threads_started":1}
I20260619 08:26:15.448910 8958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:15.448757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:50479)
0619 08:26:15.448820 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:15.448864 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 08:26:15.448905 8745 raft_consensus.cc:1203] T 8292fdcf793543ac9c19b4292c9cb8cc P ade1c097ed4646dfa7d5cb6415591387 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260619 08:26:15.449566 8963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:15.449342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:40931)
0619 08:26:15.449455 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:15.449497 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":42,"threads_started":1}
I20260619 08:26:16.309023 8964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:16.308786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:33259)
0619 08:26:16.308933 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:16.308975 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260619 08:26:16.449734 8965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:16.449424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:51907)
0619 08:26:16.449575 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:16.449667 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":67,"threads_started":1}
I20260619 08:26:16.450155 8966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:16.449910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:54979)
0619 08:26:16.450053 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:16.450093 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260619 08:26:17.309831 8967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:17.309598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:47793)
0619 08:26:17.309737 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:17.309781 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":64,"threads_started":1}
I20260619 08:26:17.450853 8968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:17.450573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:37519)
0619 08:26:17.450758 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:17.450803 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":104,"threads_started":1}
I20260619 08:26:17.450878 8969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:17.450660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:50547)
0619 08:26:17.450793 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:17.450832 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":55,"threads_started":1}
I20260619 08:26:18.310499 8970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:18.310273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:54115)
0619 08:26:18.310407 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:18.310450 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260619 08:26:18.451660 8972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:18.451410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:60505)
0619 08:26:18.451564 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:18.451606 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260619 08:26:18.451665 8971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:18.451398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:44641)
0619 08:26:18.451583 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:18.451622 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":65,"threads_started":1}
I20260619 08:26:19.311275 8974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:19.310972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:40527)
0619 08:26:19.311156 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:19.311225 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":61,"threads_started":1}
I20260619 08:26:19.452396 8975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:19.452176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:52409)
0619 08:26:19.452305 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:19.452346 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
I20260619 08:26:19.452401 8976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:19.452175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:47707)
0619 08:26:19.452326 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:19.452359 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":71,"threads_started":1}
I20260619 08:26:20.312036 8977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:20.311798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:55831)
0619 08:26:20.311948 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:20.311989 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
I20260619 08:26:20.453089 8978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:20.452847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:48499)
0619 08:26:20.452998 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:20.453042 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":61,"threads_started":1}
I20260619 08:26:20.453104 8979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:20.452874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:47635)
0619 08:26:20.453011 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:20.453047 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260619 08:26:21.312794 8980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:21.312568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:48253)
0619 08:26:21.312700 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:21.312745 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260619 08:26:21.453820 8981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:21.453544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:57959)
0619 08:26:21.453720 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:21.453765 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":96,"threads_started":1}
I20260619 08:26:21.453848 8982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:21.453657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:44635)
0619 08:26:21.453773 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:21.453807 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":64,"threads_started":1}
I20260619 08:26:22.313642 8983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:22.313385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:36871)
0619 08:26:22.313529 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:22.313592 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260619 08:26:22.454592 8984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:22.454322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:49275)
0619 08:26:22.454460 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:22.454508 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
I20260619 08:26:22.454610 8985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:22.454402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:38731)
0619 08:26:22.454527 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:22.454569 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260619 08:26:23.314407 8986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:23.314178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:44767)
0619 08:26:23.314313 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:23.314360 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260619 08:26:23.455351 8987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:23.455121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:58851)
0619 08:26:23.455264 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:23.455302 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260619 08:26:23.455359 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:23.455157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:33983)
0619 08:26:23.455285 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:23.455317 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260619 08:26:24.315210 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:24.314976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:38731)
0619 08:26:24.315116 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:24.315155 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260619 08:26:24.456065 8991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:24.455840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:47135)
0619 08:26:24.455978 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:24.456016 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260619 08:26:24.456065 8990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:24.455840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:46367)
0619 08:26:24.455991 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:24.456017 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260619 08:26:25.316222 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:25.315844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:47771)
0619 08:26:25.316061 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:25.316113 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":123,"threads_started":1}
I20260619 08:26:25.456844 8993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:25.456573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:43659)
0619 08:26:25.456751 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:25.456795 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":78,"threads_started":1}
I20260619 08:26:25.456844 8994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:25.456573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:35193)
0619 08:26:25.456763 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:25.456797 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":79,"threads_started":1}
I20260619 08:26:26.316955 8995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:26.316739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.131:48557)
0619 08:26:26.316864 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:26.316908 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
I20260619 08:26:26.457713 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:26.457443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.129:49077)
0619 08:26:26.457626 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:26.457662 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":84,"threads_started":1}
I20260619 08:26:26.457715 8996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:26.457443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:34081 (local address 127.25.127.130:51689)
0619 08:26:26.457620 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:26.457669 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:34081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":79,"threads_started":1}
I20260619 08:26:27.247522 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 8497
I20260619 08:26:27.252310 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 8662
I20260619 08:26:27.257318 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 8794
2026-06-19T08:26:27Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18299 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:26:27 dist-test-slave-5rhg krb5kdc[9000](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:26:27 dist-test-slave-5rhg krb5kdc[9000](info): set up 2 sockets
Jun 19 08:26:27 dist-test-slave-5rhg krb5kdc[9000](info): commencing operation
krb5kdc: starting...
W20260619 08:26:28.309158 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:26:28 dist-test-slave-5rhg krb5kdc[9000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:26:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:26:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:26:28.425288 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:46391
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:46391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:26:28.506464 9013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:28.506731 9013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:28.506772 9013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:28.507989 9013 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 08:26:28.508064 9013 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 08:26:28.508085 9013 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 08:26:28.508103 9013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:26:28.508131 9013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:28.508196 9013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:28.508247 9013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:26:28.508272 9013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:26:28.509841 9013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:46391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:46391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:28.510138 9013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:28.510380 9013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:28.515815 9018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:28.515895 9013 server_base.cc:1034] running on GCE node
W20260619 08:26:28.515843 9022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:28.515828 9019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:28.516353 9013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:28.516644 9013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:28.517836 9013 hybrid_clock.cc:648] HybridClock initialized: now 1781857588517812 us; error 36 us; skew 500 ppm
Jun 19 08:26:28 dist-test-slave-5rhg krb5kdc[9000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857588, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:28.520614 9013 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:26:28.521070 9013 webserver.cc:460] Webserver started at http://127.25.127.190:42677/ using document root <none> and password file <none>
I20260619 08:26:28.521255 9013 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:28.521294 9013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:28.521392 9013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:26:28.522418 9013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "5bc56cd048ef4b628769f12a41761435"
format_stamp: "Formatted at 2026-06-19 08:26:28 on dist-test-slave-5rhg"
server_key: "b8bd1e93f9b109296eab09dcb08de45a"
server_key_iv: "14f98c9283c72798181ded1463335786"
server_key_version: "encryptionkey@0"
I20260619 08:26:28.522725 9013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "5bc56cd048ef4b628769f12a41761435"
format_stamp: "Formatted at 2026-06-19 08:26:28 on dist-test-slave-5rhg"
server_key: "b8bd1e93f9b109296eab09dcb08de45a"
server_key_iv: "14f98c9283c72798181ded1463335786"
server_key_version: "encryptionkey@0"
I20260619 08:26:28.523989 9013 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 08:26:28.524808 9029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:28.524973 9013 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:28.525056 9013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "5bc56cd048ef4b628769f12a41761435"
format_stamp: "Formatted at 2026-06-19 08:26:28 on dist-test-slave-5rhg"
server_key: "b8bd1e93f9b109296eab09dcb08de45a"
server_key_iv: "14f98c9283c72798181ded1463335786"
server_key_version: "encryptionkey@0"
I20260619 08:26:28.525115 9013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:28.536795 9013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:28.539080 9013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:28.539194 9013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:28.542989 9013 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:46391
I20260619 08:26:28.543043 9081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:46391 every 8 connection(s)
I20260619 08:26:28.543365 9013 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:26:28.544305 9082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:28.546627 9082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: Bootstrap starting.
I20260619 08:26:28.547252 9082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:28.547567 9082 log.cc:824] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:28.548364 9082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: No bootstrap required, opened a new log
I20260619 08:26:28.549860 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 9013
I20260619 08:26:28.549939 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:26:28.550137 26110 external_mini_cluster.cc:1428] Setting key 929734b9d39b2303448123f69aa7ce70
I20260619 08:26:28.550041 9082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc56cd048ef4b628769f12a41761435" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46391 } }
I20260619 08:26:28.550194 9082 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:28.550246 9082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bc56cd048ef4b628769f12a41761435, State: Initialized, Role: FOLLOWER
I20260619 08:26:28.550427 9082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc56cd048ef4b628769f12a41761435" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46391 } }
I20260619 08:26:28.550520 9082 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:26:28.550554 9082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:26:28.550603 9082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:28.551774 9082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc56cd048ef4b628769f12a41761435" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46391 } }
I20260619 08:26:28.551914 9082 leader_election.cc:302] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bc56cd048ef4b628769f12a41761435; no voters:
I20260619 08:26:28.552096 9082 leader_election.cc:288] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:26:28.552137 9087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:26:28.552328 9087 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 1 LEADER]: Becoming Leader. State: Replica: 5bc56cd048ef4b628769f12a41761435, State: Running, Role: LEADER
I20260619 08:26:28.552426 9082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:26:28.552482 9087 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc56cd048ef4b628769f12a41761435" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46391 } }
I20260619 08:26:28.552964 9088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bc56cd048ef4b628769f12a41761435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc56cd048ef4b628769f12a41761435" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46391 } } }
I20260619 08:26:28.553048 9088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [sys.catalog]: This master's current role is: LEADER
I20260619 08:26:28.553254 9089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bc56cd048ef4b628769f12a41761435. Latest consensus state: current_term: 1 leader_uuid: "5bc56cd048ef4b628769f12a41761435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc56cd048ef4b628769f12a41761435" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46391 } } }
I20260619 08:26:28.553346 9089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [sys.catalog]: This master's current role is: LEADER
I20260619 08:26:28.553870 9095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:26:28.554528 9095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:26:28.556140 9095 catalog_manager.cc:1347] Generated new cluster ID: 4da34608d19742fc9f0b5eb9299aa48b
I20260619 08:26:28.556216 9095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:26:28 dist-test-slave-5rhg krb5kdc[9000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:28.561196 9095 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:26:28.561936 9095 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:26:28.562551 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:28.550633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38830 (local address 127.25.127.190:46391)
0619 08:26:28.550835 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:28.550845 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:28.550886 (+ 41us) server_negotiation.cc:407] Connection header received
0619 08:26:28.551040 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:28.551047 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:28.551104 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:28.551239 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:28.552107 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:28.553526 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:28.554679 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:28.555345 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:28.555512 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:28.558485 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:28.558489 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:28.558490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:28.558503 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:28.560452 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:28.560950 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:28.560953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:28.560954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:28.561081 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:28.561345 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:28.561346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:28.561347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:28.561621 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:28.561743 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:28.562201 (+ 458us) server_negotiation.cc:299] Negotiation successful
0619 08:26:28.562351 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":100,"threads_started":1}
I20260619 08:26:28.570397 9095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: Generated new TSK 0
I20260619 08:26:28.570611 9095 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
I20260619 08:26:28.601133 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:46391
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:26:28.684429 9108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:28.684643 9108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:28.684666 9108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:28.685920 9108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:28.685962 9108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:28.685989 9108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:26:28.687453 9108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:46391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:28.687705 9108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:28.687943 9108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:28.693145 9116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:28.693177 9113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:28.693145 9114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:28.693631 9108 server_base.cc:1034] running on GCE node
I20260619 08:26:28.693822 9108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:28.694048 9108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:28.695189 9108 hybrid_clock.cc:648] HybridClock initialized: now 1781857588695175 us; error 29 us; skew 500 ppm
Jun 19 08:26:28 dist-test-slave-5rhg krb5kdc[9000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857588, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:28.697870 9108 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:26:28.698344 9108 webserver.cc:460] Webserver started at http://127.25.127.129:37909/ using document root <none> and password file <none>
I20260619 08:26:28.698596 9108 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:28.698665 9108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:28.698812 9108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:26:28.699880 9108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "982f047f6beb412c87a89d625f6e5292"
format_stamp: "Formatted at 2026-06-19 08:26:28 on dist-test-slave-5rhg"
server_key: "d04fc56c66e8b751d333e8662287910b"
server_key_iv: "d1fe2fab7ed278c2a5b47dc864a84a84"
server_key_version: "encryptionkey@0"
I20260619 08:26:28.700325 9108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "982f047f6beb412c87a89d625f6e5292"
format_stamp: "Formatted at 2026-06-19 08:26:28 on dist-test-slave-5rhg"
server_key: "d04fc56c66e8b751d333e8662287910b"
server_key_iv: "d1fe2fab7ed278c2a5b47dc864a84a84"
server_key_version: "encryptionkey@0"
I20260619 08:26:28.701664 9108 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:26:28.702390 9123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:28.702571 9108 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:28.702669 9108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "982f047f6beb412c87a89d625f6e5292"
format_stamp: "Formatted at 2026-06-19 08:26:28 on dist-test-slave-5rhg"
server_key: "d04fc56c66e8b751d333e8662287910b"
server_key_iv: "d1fe2fab7ed278c2a5b47dc864a84a84"
server_key_version: "encryptionkey@0"
I20260619 08:26:28.702754 9108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:28.729023 9108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:28.731057 9108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:28.731233 9108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:28.731485 9108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:28.731830 9108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:26:28.731886 9108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:28.731937 9108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:26:28.731964 9108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:28.737871 9108 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:33843
I20260619 08:26:28.737950 9237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:33843 every 8 connection(s)
I20260619 08:26:28.738297 9108 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:26:28 dist-test-slave-5rhg krb5kdc[9000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857588, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:28.746315 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 9108
I20260619 08:26:28.746431 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:26:28.746708 26110 external_mini_cluster.cc:1428] Setting key fa65ef464cc29d7bf919c24c08adbb21
I20260619 08:26:28.747781 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:28.739107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:54809 (local address 127.25.127.190:46391)
0619 08:26:28.739179 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:28.739181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:28.739374 (+ 193us) server_negotiation.cc:407] Connection header received
0619 08:26:28.739857 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:28.739859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:28.739868 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:28.739941 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:28.740990 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:28.741516 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:28.742501 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:28.742689 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:28.742740 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:28.744875 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:28.744876 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:28.744876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:28.744883 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:28.746168 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:28.746666 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:28.746667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:28.746668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:28.746725 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:28.747074 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:28.747075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:28.747076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:28.747210 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:28.747278 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:28.747644 (+ 366us) server_negotiation.cc:299] Negotiation successful
0619 08:26:28.747717 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:26:28.748106 9240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:28.739069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:54809)
0619 08:26:28.739238 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:28.739280 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:28.739774 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:28.740013 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:28.740017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:28.740683 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:28.740886 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:28.740891 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:28.741603 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:28.741604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:28.742424 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:28.742426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:28.742761 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:28.742761 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:28.742864 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:28.743346 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:28.743355 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:28.744792 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:28.746250 (+ 1458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:28.746260 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:28.746262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:28.746589 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:28.746828 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:28.746829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:28.746830 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:28.746997 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:28.747275 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:28.747280 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:28.747480 (+ 200us) client_negotiation.cc:769] Sending connection context
0619 08:26:28.747601 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 08:26:28.747786 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260619 08:26:28.748637 9238 heartbeater.cc:347] Connected to a master server at 127.25.127.190:46391
I20260619 08:26:28.748744 9238 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:28.748961 9238 heartbeater.cc:511] Master 127.25.127.190:46391 requested a full tablet report, sending...
I20260619 08:26:28.749547 9046 ts_manager.cc:194] Registered new tserver with Master: 982f047f6beb412c87a89d625f6e5292 (127.25.127.129:33843)
I20260619 08:26:28.750357 9046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:54809
WARNING: no policy specified for kudu/127.25.127.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
I20260619 08:26:28.773293 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:46391
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:26:28.857118 9243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:28.857404 9243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:28.857442 9243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:28.859565 9243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:28.859647 9243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:28.859716 9243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:26:28.862226 9243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:46391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:28.862552 9243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:28.862808 9243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:28.868361 9251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:28.868412 9248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:28.868572 9243 server_base.cc:1034] running on GCE node
W20260619 08:26:28.868376 9249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:28.868889 9243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:28.869149 9243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:28.870322 9243 hybrid_clock.cc:648] HybridClock initialized: now 1781857588870297 us; error 35 us; skew 500 ppm
Jun 19 08:26:28 dist-test-slave-5rhg krb5kdc[9000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857588, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:28.872958 9243 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:26:28.873415 9243 webserver.cc:460] Webserver started at http://127.25.127.130:40389/ using document root <none> and password file <none>
I20260619 08:26:28.873656 9243 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:28.873728 9243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:28.873872 9243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:26:28.874943 9243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "974b12b77cec4cc2827d3f40ca922c43"
format_stamp: "Formatted at 2026-06-19 08:26:28 on dist-test-slave-5rhg"
server_key: "226177830e922cf1980d6a6a4a557d4c"
server_key_iv: "57f18755e9f014342387c7159bdd61fe"
server_key_version: "encryptionkey@0"
I20260619 08:26:28.875339 9243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "974b12b77cec4cc2827d3f40ca922c43"
format_stamp: "Formatted at 2026-06-19 08:26:28 on dist-test-slave-5rhg"
server_key: "226177830e922cf1980d6a6a4a557d4c"
server_key_iv: "57f18755e9f014342387c7159bdd61fe"
server_key_version: "encryptionkey@0"
I20260619 08:26:28.876816 9243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260619 08:26:28.877641 9258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:28.878046 9243 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:28.878141 9243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "974b12b77cec4cc2827d3f40ca922c43"
format_stamp: "Formatted at 2026-06-19 08:26:28 on dist-test-slave-5rhg"
server_key: "226177830e922cf1980d6a6a4a557d4c"
server_key_iv: "57f18755e9f014342387c7159bdd61fe"
server_key_version: "encryptionkey@0"
I20260619 08:26:28.878227 9243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:28.896726 9243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:28.898844 9243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:28.899024 9243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:28.899272 9243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:28.899601 9243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:26:28.899654 9243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:28.899695 9243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:26:28.899727 9243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:28.906008 9243 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:33433
I20260619 08:26:28.906092 9372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:33433 every 8 connection(s)
I20260619 08:26:28.906426 9243 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:26:28.908653 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 9243
I20260619 08:26:28.908764 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:26:28.908993 26110 external_mini_cluster.cc:1428] Setting key 084b5da924b806dbb2274040607f5766
Jun 19 08:26:28 dist-test-slave-5rhg krb5kdc[9000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857588, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:28.917066 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:28.907194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:46747 (local address 127.25.127.190:46391)
0619 08:26:28.907277 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:28.907279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:28.907440 (+ 161us) server_negotiation.cc:407] Connection header received
0619 08:26:28.907964 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:28.907965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:28.907976 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:28.908049 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:28.909506 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:28.910019 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:28.911051 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:28.911263 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:28.911352 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:28.914068 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:28.914070 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:28.914070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:28.914078 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:28.915542 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:28.916058 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:28.916059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:28.916059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:28.916110 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:28.916399 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:28.916400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:28.916401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:28.916525 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:28.916600 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:28.916879 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 08:26:28.916972 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 08:26:28.917275 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:28.907124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:46747)
0619 08:26:28.907298 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:28.907341 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:28.907843 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:28.908129 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:28.908133 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:28.909068 (+ 935us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:28.909403 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:28.909418 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:28.910128 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:28.910129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:28.910953 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:28.910956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:28.911372 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:28.911373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:28.911514 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:28.912070 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:28.912078 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:28.913912 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:28.915629 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:28.915637 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:28.915639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:28.915981 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:28.916204 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:28.916205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:28.916206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:28.916328 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:28.916592 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:28.916599 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:28.916746 (+ 147us) client_negotiation.cc:769] Sending connection context
0619 08:26:28.916858 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 08:26:28.917028 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
WARNING: no policy specified for kudu/127.25.127.131@KRBTEST.COM; defaulting to no policy
I20260619 08:26:28.917914 9373 heartbeater.cc:347] Connected to a master server at 127.25.127.190:46391
I20260619 08:26:28.918031 9373 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:28.918215 9373 heartbeater.cc:511] Master 127.25.127.190:46391 requested a full tablet report, sending...
I20260619 08:26:28.918640 9046 ts_manager.cc:194] Registered new tserver with Master: 974b12b77cec4cc2827d3f40ca922c43 (127.25.127.130:33433)
I20260619 08:26:28.919029 9046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:46747
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
I20260619 08:26:28.934320 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:46391
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:26:29.017818 9378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:29.018039 9378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:29.018065 9378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:29.019245 9378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:29.019292 9378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:29.019320 9378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:26:29.020867 9378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:46391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:29.021075 9378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:29.021258 9378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:29.026544 9384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:29.026692 9378 server_base.cc:1034] running on GCE node
W20260619 08:26:29.026594 9383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:29.026568 9386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:29.026985 9378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:29.027274 9378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:29.028445 9378 hybrid_clock.cc:648] HybridClock initialized: now 1781857589028424 us; error 40 us; skew 500 ppm
Jun 19 08:26:29 dist-test-slave-5rhg krb5kdc[9000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857589, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:29.031183 9378 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:26:29.031634 9378 webserver.cc:460] Webserver started at http://127.25.127.131:35069/ using document root <none> and password file <none>
I20260619 08:26:29.031891 9378 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:29.031939 9378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:29.032083 9378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:26:29.033221 9378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "d8399394bb6f482db4248d9b9c51fa49"
format_stamp: "Formatted at 2026-06-19 08:26:29 on dist-test-slave-5rhg"
server_key: "d162f45ec8b3eb89f1ea25d52bec2012"
server_key_iv: "169a135f1215113c001efa49678593c7"
server_key_version: "encryptionkey@0"
I20260619 08:26:29.033587 9378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "d8399394bb6f482db4248d9b9c51fa49"
format_stamp: "Formatted at 2026-06-19 08:26:29 on dist-test-slave-5rhg"
server_key: "d162f45ec8b3eb89f1ea25d52bec2012"
server_key_iv: "169a135f1215113c001efa49678593c7"
server_key_version: "encryptionkey@0"
I20260619 08:26:29.035001 9378 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:26:29.035784 9393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:29.035995 9378 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:29.036087 9378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "d8399394bb6f482db4248d9b9c51fa49"
format_stamp: "Formatted at 2026-06-19 08:26:29 on dist-test-slave-5rhg"
server_key: "d162f45ec8b3eb89f1ea25d52bec2012"
server_key_iv: "169a135f1215113c001efa49678593c7"
server_key_version: "encryptionkey@0"
I20260619 08:26:29.036196 9378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:29.046133 9378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:29.047999 9378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:29.048157 9378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:29.048442 9378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:29.048767 9378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:26:29.048800 9378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:29.048861 9378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:26:29.048895 9378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:29.055167 9378 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:45615
I20260619 08:26:29.055230 9507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:45615 every 8 connection(s)
I20260619 08:26:29.055569 9378 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:26:29.059540 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 9378
I20260619 08:26:29.059635 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:26:29.059840 26110 external_mini_cluster.cc:1428] Setting key fb48de74e299c1a3dbc00fff01c60a38
Jun 19 08:26:29 dist-test-slave-5rhg krb5kdc[9000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857589, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:29.065533 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:29.056386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:34723 (local address 127.25.127.190:46391)
0619 08:26:29.056469 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:29.056472 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:29.056666 (+ 194us) server_negotiation.cc:407] Connection header received
0619 08:26:29.057150 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:29.057151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:29.057162 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:29.057213 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:29.058119 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.058617 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.059756 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.060065 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.060136 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:29.062740 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:29.062741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:29.062742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:29.062750 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:29.064088 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:29.064540 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:29.064542 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:29.064542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:29.064610 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:29.064878 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:29.064879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:29.064880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:29.065003 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:29.065081 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:29.065351 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 08:26:29.065442 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 08:26:29.065724 9510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:29.056362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:34723)
0619 08:26:29.056544 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:29.056581 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:29.057071 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:29.057265 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:29.057268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:29.057843 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:29.058040 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:29.058044 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.058721 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.058722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:29.059675 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:29.059678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.060212 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.060213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:29.060423 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:29.061070 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:29.061078 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:29.062640 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:29.064207 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:29.064214 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:29.064217 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:29.064449 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:29.064684 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:29.064685 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:29.064685 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:29.064797 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:29.065071 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:29.065077 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:29.065218 (+ 141us) client_negotiation.cc:769] Sending connection context
0619 08:26:29.065328 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 08:26:29.065488 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":98,"threads_started":1}
I20260619 08:26:29.066234 9508 heartbeater.cc:347] Connected to a master server at 127.25.127.190:46391
I20260619 08:26:29.066322 9508 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:29.066522 9508 heartbeater.cc:511] Master 127.25.127.190:46391 requested a full tablet report, sending...
I20260619 08:26:29.066934 9046 ts_manager.cc:194] Registered new tserver with Master: d8399394bb6f482db4248d9b9c51fa49 (127.25.127.131:45615)
I20260619 08:26:29.067317 9046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:34723
I20260619 08:26:29.071646 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:26:29.079121 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:29.073138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38846 (local address 127.25.127.190:46391)
0619 08:26:29.073211 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:29.073214 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:29.073260 (+ 46us) server_negotiation.cc:407] Connection header received
0619 08:26:29.073305 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:29.073306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:29.073315 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:29.073362 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:29.073953 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.074464 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.075213 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.075403 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.075462 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:29.076705 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:29.076706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:29.076707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:29.076712 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:29.078133 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:29.078494 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:29.078495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:29.078495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:29.078546 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:29.078695 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:29.078695 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:29.078696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:29.078823 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:29.078890 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:29.078968 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 08:26:29.079024 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 08:26:29.080732 9046 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:26:29.082470 9046 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:26:29.089011 9520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:29.084686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:33843 (local address 127.0.0.1:40926)
0619 08:26:29.084853 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:29.084897 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:29.084962 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:29.085368 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:29.085369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:29.085371 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:29.085440 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:29.085442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.086756 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.086757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:29.087789 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:29.087790 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.088762 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.088764 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:29.088828 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:29.088830 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:26:29.088892 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 08:26:29.088937 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":11,"thread_start_us":67,"threads_started":1}
I20260619 08:26:29.089043 9521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:29.084793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40926 (local address 127.25.127.129:33843)
0619 08:26:29.084946 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:29.084952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:29.084964 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:26:29.085254 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:29.085264 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:29.085287 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:29.085361 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:29.085523 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.086651 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.087894 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.088674 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.088762 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:29.088820 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:29.088908 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 08:26:29.088954 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260619 08:26:29.088918 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:29.084702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:45615 (local address 127.0.0.1:36642)
0619 08:26:29.085107 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:29.085118 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:29.085162 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:29.085352 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:29.085355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:29.085358 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:29.085479 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:29.085482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.086434 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.086434 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:29.087666 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:29.087668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.088557 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.088558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:29.088651 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:29.088664 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 08:26:29.088765 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 08:26:29.088819 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":89,"threads_started":1}
I20260619 08:26:29.089355 9523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:29.084986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36642 (local address 127.25.127.131:45615)
0619 08:26:29.085156 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:29.085161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:29.085172 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:26:29.085232 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:29.085234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:29.085251 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:29.085344 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:29.085561 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.086301 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.087748 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.088457 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.088557 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:29.088617 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:29.088984 (+ 367us) server_negotiation.cc:299] Negotiation successful
0619 08:26:29.089030 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260619 08:26:29.089473 9524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:29.085113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47404 (local address 127.25.127.130:33433)
0619 08:26:29.085500 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:29.085504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:29.085513 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:26:29.085552 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:29.085555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:29.085571 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:29.085639 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:29.085846 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.086866 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.088029 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.088709 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.088779 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:29.088824 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:29.089342 (+ 518us) server_negotiation.cc:299] Negotiation successful
0619 08:26:29.089391 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":81,"threads_started":1}
I20260619 08:26:29.089598 9519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:29.084896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:33433 (local address 127.0.0.1:47404)
0619 08:26:29.085164 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:29.085176 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:29.085230 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:29.085719 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:29.085720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:29.085723 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:29.085756 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:29.085758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.086967 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.086968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:29.087938 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:29.087941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.088858 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.088860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:29.088934 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:29.088938 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:26:29.088988 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 08:26:29.089027 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":67,"threads_started":1}
I20260619 08:26:29.089775 9172 tablet_service.cc:1467] Processing CreateTablet for tablet ff4350c8efb74c05b86a834afee45266 (DEFAULT_TABLE table=test-table [id=c82873562a8f4873a8bd41791eb8ac3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:26:29.090019 9442 tablet_service.cc:1467] Processing CreateTablet for tablet ff4350c8efb74c05b86a834afee45266 (DEFAULT_TABLE table=test-table [id=c82873562a8f4873a8bd41791eb8ac3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:26:29.090139 9172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff4350c8efb74c05b86a834afee45266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:29.090230 9306 tablet_service.cc:1467] Processing CreateTablet for tablet ff4350c8efb74c05b86a834afee45266 (DEFAULT_TABLE table=test-table [id=c82873562a8f4873a8bd41791eb8ac3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:26:29.090354 9442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff4350c8efb74c05b86a834afee45266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:29.090516 9306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff4350c8efb74c05b86a834afee45266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:29.093133 9525 tablet_bootstrap.cc:492] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49: Bootstrap starting.
I20260619 08:26:29.093847 9525 tablet_bootstrap.cc:654] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:29.093871 9527 tablet_bootstrap.cc:492] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43: Bootstrap starting.
I20260619 08:26:29.094353 9527 tablet_bootstrap.cc:654] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:29.094353 9525 log.cc:824] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:29.094887 9527 log.cc:824] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:29.095180 9525 tablet_bootstrap.cc:492] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49: No bootstrap required, opened a new log
I20260619 08:26:29.095217 9526 tablet_bootstrap.cc:492] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: Bootstrap starting.
I20260619 08:26:29.095260 9525 ts_tablet_manager.cc:1397] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:26:29.095719 9526 tablet_bootstrap.cc:654] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:29.096269 9526 log.cc:824] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:29.096817 9527 tablet_bootstrap.cc:492] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43: No bootstrap required, opened a new log
I20260619 08:26:29.096870 9527 ts_tablet_manager.cc:1397] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 08:26:29.097087 9525 raft_consensus.cc:348] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } } peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } }
I20260619 08:26:29.097249 9525 raft_consensus.cc:374] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:29.097294 9525 raft_consensus.cc:729] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8399394bb6f482db4248d9b9c51fa49, State: Initialized, Role: FOLLOWER
I20260619 08:26:29.097451 9525 consensus_queue.cc:260] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } } peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } }
I20260619 08:26:29.097741 9525 ts_tablet_manager.cc:1428] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:26:29.097882 9508 heartbeater.cc:503] Master 127.25.127.190:46391 was elected leader, sending a full tablet report...
I20260619 08:26:29.098830 9527 raft_consensus.cc:348] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } } peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } }
I20260619 08:26:29.099025 9527 raft_consensus.cc:374] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:29.099072 9527 raft_consensus.cc:729] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 974b12b77cec4cc2827d3f40ca922c43, State: Initialized, Role: FOLLOWER
I20260619 08:26:29.099195 9527 consensus_queue.cc:260] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } } peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } }
I20260619 08:26:29.099454 9527 ts_tablet_manager.cc:1428] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:26:29.099478 9373 heartbeater.cc:503] Master 127.25.127.190:46391 was elected leader, sending a full tablet report...
I20260619 08:26:29.099799 9526 tablet_bootstrap.cc:492] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: No bootstrap required, opened a new log
I20260619 08:26:29.099879 9526 ts_tablet_manager.cc:1397] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260619 08:26:29.101512 9526 raft_consensus.cc:348] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } } peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } }
I20260619 08:26:29.101667 9526 raft_consensus.cc:374] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:29.101714 9526 raft_consensus.cc:729] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982f047f6beb412c87a89d625f6e5292, State: Initialized, Role: FOLLOWER
I20260619 08:26:29.101824 9526 consensus_queue.cc:260] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } } peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } }
I20260619 08:26:29.102093 9526 ts_tablet_manager.cc:1428] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260619 08:26:29.102155 9238 heartbeater.cc:503] Master 127.25.127.190:46391 was elected leader, sending a full tablet report...
W20260619 08:26:29.157047 9374 tablet.cc:2369] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:26:29.239075 9239 tablet.cc:2369] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:26:29.294180 9531 raft_consensus.cc:482] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:26:29.294313 9531 raft_consensus.cc:504] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } } peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } }
I20260619 08:26:29.294659 9531 leader_election.cc:288] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 982f047f6beb412c87a89d625f6e5292 (127.25.127.129:33843), 974b12b77cec4cc2827d3f40ca922c43 (127.25.127.130:33433)
I20260619 08:26:29.298754 9524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:29.294939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:45189 (local address 127.25.127.130:33433)
0619 08:26:29.294988 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:29.294991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:29.295047 (+ 56us) server_negotiation.cc:407] Connection header received
0619 08:26:29.295075 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:29.295076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:29.295103 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:29.295167 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:29.295372 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.296239 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.297783 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.298361 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.298431 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:29.298440 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:29.298634 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 08:26:29.298684 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:26:29.298754 9521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:29.294939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:39295 (local address 127.25.127.129:33843)
0619 08:26:29.294988 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:29.294991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:29.295127 (+ 136us) server_negotiation.cc:407] Connection header received
0619 08:26:29.295158 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:29.295159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:29.295174 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:29.295236 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:29.295371 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.296237 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.297769 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.298343 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.298430 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:29.298439 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:29.298634 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 08:26:29.298683 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:26:29.298753 9510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:29.294852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:33433 (local address 127.25.127.131:45189)
0619 08:26:29.294925 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:29.294938 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:29.294986 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:29.295181 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:29.295181 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:29.295183 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:29.295276 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:29.295278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.296327 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.296328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:29.297658 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:29.297661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.298453 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.298454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:29.298525 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:29.298528 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:26:29.298580 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:26:29.298634 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":73}
I20260619 08:26:29.298758 9534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:29.294860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:33843 (local address 127.25.127.131:39295)
0619 08:26:29.295033 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:29.295041 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:29.295092 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:29.295239 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:29.295240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:29.295241 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:29.295298 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:29.295299 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.296334 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.296334 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:29.297658 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:29.297661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.298437 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.298439 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:29.298513 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:29.298517 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:26:29.298580 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 08:26:29.298633 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":75,"thread_start_us":103,"threads_started":1}
I20260619 08:26:29.299158 9326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff4350c8efb74c05b86a834afee45266" candidate_uuid: "d8399394bb6f482db4248d9b9c51fa49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "974b12b77cec4cc2827d3f40ca922c43" is_pre_election: true
I20260619 08:26:29.299178 9192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff4350c8efb74c05b86a834afee45266" candidate_uuid: "d8399394bb6f482db4248d9b9c51fa49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982f047f6beb412c87a89d625f6e5292" is_pre_election: true
I20260619 08:26:29.299336 9326 raft_consensus.cc:2454] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8399394bb6f482db4248d9b9c51fa49 in term 0.
I20260619 08:26:29.299336 9192 raft_consensus.cc:2454] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8399394bb6f482db4248d9b9c51fa49 in term 0.
I20260619 08:26:29.299590 9397 leader_election.cc:302] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 974b12b77cec4cc2827d3f40ca922c43, d8399394bb6f482db4248d9b9c51fa49; no voters:
I20260619 08:26:29.299803 9531 raft_consensus.cc:2781] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:26:29.299919 9531 raft_consensus.cc:482] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:26:29.299959 9531 raft_consensus.cc:3037] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:29.300779 9531 raft_consensus.cc:504] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } } peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } }
I20260619 08:26:29.300976 9531 leader_election.cc:288] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [CANDIDATE]: Term 1 election: Requested vote from peers 982f047f6beb412c87a89d625f6e5292 (127.25.127.129:33843), 974b12b77cec4cc2827d3f40ca922c43 (127.25.127.130:33433)
I20260619 08:26:29.301162 9192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff4350c8efb74c05b86a834afee45266" candidate_uuid: "d8399394bb6f482db4248d9b9c51fa49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982f047f6beb412c87a89d625f6e5292"
I20260619 08:26:29.301221 9326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff4350c8efb74c05b86a834afee45266" candidate_uuid: "d8399394bb6f482db4248d9b9c51fa49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "974b12b77cec4cc2827d3f40ca922c43"
I20260619 08:26:29.301249 9192 raft_consensus.cc:3037] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:29.301355 9326 raft_consensus.cc:3037] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:29.302009 9192 raft_consensus.cc:2454] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8399394bb6f482db4248d9b9c51fa49 in term 1.
I20260619 08:26:29.302127 9326 raft_consensus.cc:2454] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8399394bb6f482db4248d9b9c51fa49 in term 1.
I20260619 08:26:29.302181 9396 leader_election.cc:302] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 982f047f6beb412c87a89d625f6e5292, d8399394bb6f482db4248d9b9c51fa49; no voters:
I20260619 08:26:29.302292 9531 raft_consensus.cc:2781] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:26:29.302495 9531 raft_consensus.cc:686] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [term 1 LEADER]: Becoming Leader. State: Replica: d8399394bb6f482db4248d9b9c51fa49, State: Running, Role: LEADER
I20260619 08:26:29.302603 9531 consensus_queue.cc:237] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } } peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } }
I20260619 08:26:29.303509 9046 catalog_manager.cc:5495] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 reported cstate change: term changed from 0 to 1, leader changed from <none> to d8399394bb6f482db4248d9b9c51fa49 (127.25.127.131). New cstate: current_term: 1 leader_uuid: "d8399394bb6f482db4248d9b9c51fa49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } health_report { overall_health: UNKNOWN } } }
W20260619 08:26:29.306134 9509 tablet.cc:2369] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:26:29.314546 9523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:29.312309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36644 (local address 127.25.127.131:45615)
0619 08:26:29.312377 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:29.312379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:29.312388 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:26:29.312428 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:29.312429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:29.312449 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:29.312516 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:26:29.312669 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.313142 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.313896 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:29.314076 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:29.314124 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:29.314262 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:26:29.314359 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:26:29.314421 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:29.314452 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 08:26:29.314470 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:26:29.316417 9326 raft_consensus.cc:1261] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43 [term 1 FOLLOWER]: Refusing update from remote peer d8399394bb6f482db4248d9b9c51fa49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:26:29.316465 9192 raft_consensus.cc:1261] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Refusing update from remote peer d8399394bb6f482db4248d9b9c51fa49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:26:29.316772 9535 consensus_queue.cc:1035] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:26:29.316915 9531 consensus_queue.cc:1035] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:26:29.321681 9045 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:26:29.321781 9045 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:26:29.322531 9045 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: Sending DeleteTablet for 3 replicas of tablet ff4350c8efb74c05b86a834afee45266
I20260619 08:26:29.322868 9172 tablet_service.cc:1514] Processing DeleteTablet for tablet ff4350c8efb74c05b86a834afee45266 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:26:29 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:40926
I20260619 08:26:29.322902 9306 tablet_service.cc:1514] Processing DeleteTablet for tablet ff4350c8efb74c05b86a834afee45266 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:26:29 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:47404
I20260619 08:26:29.323119 9548 tablet_replica.cc:330] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: stopping tablet replica
I20260619 08:26:29.323140 9549 tablet_replica.cc:330] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43: stopping tablet replica
I20260619 08:26:29.323156 9442 tablet_service.cc:1514] Processing DeleteTablet for tablet ff4350c8efb74c05b86a834afee45266 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:26:29 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:36642
I20260619 08:26:29.323236 9549 raft_consensus.cc:2229] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:26:29.323253 9548 raft_consensus.cc:2229] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:26:29.323336 9549 raft_consensus.cc:2258] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:26:29.323345 9548 raft_consensus.cc:2258] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:26:29.323586 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 9108
I20260619 08:26:29.323823 9548 ts_tablet_manager.cc:1905] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:26:29.323823 9549 ts_tablet_manager.cc:1905] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:26:29.324204 9550 tablet_replica.cc:330] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49: stopping tablet replica
I20260619 08:26:29.324295 9550 raft_consensus.cc:2229] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:26:29.324379 9550 raft_consensus.cc:2258] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:26:29.324831 9550 ts_tablet_manager.cc:1905] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:26:29.325791 9549 ts_tablet_manager.cc:1918] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:26:29.325862 9549 log.cc:1196] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/ff4350c8efb74c05b86a834afee45266
I20260619 08:26:29.326153 9549 ts_tablet_manager.cc:1939] T ff4350c8efb74c05b86a834afee45266 P 974b12b77cec4cc2827d3f40ca922c43: Deleting consensus metadata
I20260619 08:26:29.326612 9550 ts_tablet_manager.cc:1918] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:26:29.326660 9550 log.cc:1196] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/ff4350c8efb74c05b86a834afee45266
I20260619 08:26:29.326845 9550 ts_tablet_manager.cc:1939] T ff4350c8efb74c05b86a834afee45266 P d8399394bb6f482db4248d9b9c51fa49: Deleting consensus metadata
I20260619 08:26:29.331657 9033 catalog_manager.cc:4841] TS 974b12b77cec4cc2827d3f40ca922c43 (127.25.127.130:33433): tablet ff4350c8efb74c05b86a834afee45266 (table test-table [id=c82873562a8f4873a8bd41791eb8ac3a]) successfully deleted
I20260619 08:26:29.331804 9033 catalog_manager.cc:4841] TS d8399394bb6f482db4248d9b9c51fa49 (127.25.127.131:45615): tablet ff4350c8efb74c05b86a834afee45266 (table test-table [id=c82873562a8f4873a8bd41791eb8ac3a]) successfully deleted
W20260619 08:26:29.331815 9032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.127.129:33843 (error 108)
I20260619 08:26:29.332054 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 9243
I20260619 08:26:29.332394 9519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:29.332273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:33843 (local address 127.0.0.1:40938)
0619 08:26:29.332317 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:29.332352 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.129:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 08:26:29.332566 9032 catalog_manager.cc:4584] TS 982f047f6beb412c87a89d625f6e5292 (127.25.127.129:33843): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff4350c8efb74c05b86a834afee45266: Network error: Client connection negotiation failed: client connection to 127.25.127.129:33843: connect: Connection refused (error 111)
I20260619 08:26:29.339709 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 9378
I20260619 08:26:29.344449 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 9013
I20260619 08:26:49.349994 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:46391
--webserver_interface=127.25.127.190
--webserver_port=42677
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:46391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:26:49.433629 9552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:49.433853 9552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:49.433877 9552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:49.435076 9552 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 08:26:49.435204 9552 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 08:26:49.435248 9552 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 08:26:49.435271 9552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:26:49.435285 9552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:49.435300 9552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:49.435312 9552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:26:49.435333 9552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:26:49.436834 9552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:46391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:46391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=42677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:49.437048 9552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:49.437232 9552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:49.446482 9561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:49.446471 9559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:49.446465 9558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:49.446610 9552 server_base.cc:1034] running on GCE node
I20260619 08:26:49.446903 9552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:49.447276 9552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:49.448493 9552 hybrid_clock.cc:648] HybridClock initialized: now 1781857609448474 us; error 38 us; skew 500 ppm
Jun 19 08:26:49 dist-test-slave-5rhg krb5kdc[9000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857609, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:49.451203 9552 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:26:49.451725 9552 webserver.cc:460] Webserver started at http://127.25.127.190:42677/ using document root <none> and password file <none>
I20260619 08:26:49.451973 9552 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:49.452067 9552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:49.453577 9552 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:26:49.454329 9569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:49.454521 9552 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:26:49.454631 9552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "5bc56cd048ef4b628769f12a41761435"
format_stamp: "Formatted at 2026-06-19 08:26:28 on dist-test-slave-5rhg"
server_key: "b8bd1e93f9b109296eab09dcb08de45a"
server_key_iv: "14f98c9283c72798181ded1463335786"
server_key_version: "encryptionkey@0"
I20260619 08:26:49.455055 9552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:49.464854 9552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:49.467247 9552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:49.467410 9552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:49.471727 9552 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:46391
I20260619 08:26:49.471767 9621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:46391 every 8 connection(s)
I20260619 08:26:49.472355 9552 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:26:49.473959 9622 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:26:49.474671 9622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: Bootstrap starting.
I20260619 08:26:49.475589 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 9552
I20260619 08:26:49.475857 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:33843
--local_ip_for_outbound_sockets=127.25.127.129
--tserver_master_addrs=127.25.127.190:46391
--webserver_port=37909
--webserver_interface=127.25.127.129
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:26:49.484138 9622 log.cc:824] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:49.486217 9622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:26:49.486567 9622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: Bootstrap complete.
I20260619 08:26:49.489053 9622 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc56cd048ef4b628769f12a41761435" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46391 } }
I20260619 08:26:49.489288 9622 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bc56cd048ef4b628769f12a41761435, State: Initialized, Role: FOLLOWER
I20260619 08:26:49.489475 9622 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc56cd048ef4b628769f12a41761435" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46391 } }
I20260619 08:26:49.489540 9622 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:26:49.489611 9622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:26:49.489679 9622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:26:49.490728 9622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc56cd048ef4b628769f12a41761435" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46391 } }
I20260619 08:26:49.490815 9622 leader_election.cc:302] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bc56cd048ef4b628769f12a41761435; no voters:
I20260619 08:26:49.491044 9622 leader_election.cc:288] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 08:26:49.491082 9626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 08:26:49.491256 9626 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 2 LEADER]: Becoming Leader. State: Replica: 5bc56cd048ef4b628769f12a41761435, State: Running, Role: LEADER
I20260619 08:26:49.491330 9622 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:26:49.491427 9626 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc56cd048ef4b628769f12a41761435" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46391 } }
I20260619 08:26:49.491720 9626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bc56cd048ef4b628769f12a41761435. Latest consensus state: current_term: 2 leader_uuid: "5bc56cd048ef4b628769f12a41761435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc56cd048ef4b628769f12a41761435" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46391 } } }
I20260619 08:26:49.491724 9627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5bc56cd048ef4b628769f12a41761435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc56cd048ef4b628769f12a41761435" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46391 } } }
I20260619 08:26:49.491979 9627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [sys.catalog]: This master's current role is: LEADER
I20260619 08:26:49.492307 9626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [sys.catalog]: This master's current role is: LEADER
I20260619 08:26:49.492406 9635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:26:49.493229 9635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:26:49.493371 9635 catalog_manager.cc:1259] Loaded cluster ID: 4da34608d19742fc9f0b5eb9299aa48b
I20260619 08:26:49.493433 9635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:26:49.494656 9635 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:26:49.494886 9635 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: Loaded TSK: 0
I20260619 08:26:49.521746 9635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: Generated new TSK 1
I20260619 08:26:49.521975 9635 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 08:26:49.567534 9624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:49.567806 9624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:49.567852 9624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:49.569121 9624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:49.569196 9624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:49.569244 9624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:26:49.570873 9624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:33843
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=37909
--tserver_master_addrs=127.25.127.190:46391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:49.571162 9624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:49.571409 9624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:49.577795 9652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:49.577808 9650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:49.577865 9624 server_base.cc:1034] running on GCE node
W20260619 08:26:49.578099 9649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:49.578351 9624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:49.578678 9624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:49.579850 9624 hybrid_clock.cc:648] HybridClock initialized: now 1781857609579830 us; error 38 us; skew 500 ppm
Jun 19 08:26:49 dist-test-slave-5rhg krb5kdc[9000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857609, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:49.582696 9624 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:26:49.583245 9624 webserver.cc:460] Webserver started at http://127.25.127.129:37909/ using document root <none> and password file <none>
I20260619 08:26:49.583503 9624 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:49.583592 9624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:49.585089 9624 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:49.585901 9659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:49.586105 9624 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:49.586199 9624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "982f047f6beb412c87a89d625f6e5292"
format_stamp: "Formatted at 2026-06-19 08:26:28 on dist-test-slave-5rhg"
server_key: "d04fc56c66e8b751d333e8662287910b"
server_key_iv: "d1fe2fab7ed278c2a5b47dc864a84a84"
server_key_version: "encryptionkey@0"
I20260619 08:26:49.586577 9624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:49.594640 9624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:49.597218 9624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:49.597404 9624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:49.597662 9624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:49.598169 9666 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:26:49.600116 9624 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:26:49.600227 9624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:26:49.600281 9624 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:26:49.600999 9624 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:26:49.601081 9624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 08:26:49.601136 9666 tablet_bootstrap.cc:492] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: Bootstrap starting.
I20260619 08:26:49.603088 9666 log.cc:824] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:49.606703 9666 tablet_bootstrap.cc:492] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:26:49.607056 9666 tablet_bootstrap.cc:492] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: Bootstrap complete.
I20260619 08:26:49.607240 9666 ts_tablet_manager.cc:1397] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260619 08:26:49.608489 9624 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:33843
I20260619 08:26:49.608541 9775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:33843 every 8 connection(s)
I20260619 08:26:49.608925 9624 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:26:49.609581 9666 raft_consensus.cc:348] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } } peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } }
I20260619 08:26:49.609962 9666 raft_consensus.cc:729] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982f047f6beb412c87a89d625f6e5292, State: Initialized, Role: FOLLOWER
I20260619 08:26:49.610103 9666 consensus_queue.cc:260] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } } peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } }
I20260619 08:26:49.610353 9666 ts_tablet_manager.cc:1428] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
I20260619 08:26:49.611078 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 9624
I20260619 08:26:49.611343 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:33433
--local_ip_for_outbound_sockets=127.25.127.130
--tserver_master_addrs=127.25.127.190:46391
--webserver_port=40389
--webserver_interface=127.25.127.130
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 08:26:49 dist-test-slave-5rhg krb5kdc[9000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857609, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:49.625674 9779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:49.609872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:46509 (local address 127.25.127.190:46391)
0619 08:26:49.610156 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:49.610167 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:49.612505 (+ 2338us) server_negotiation.cc:407] Connection header received
0619 08:26:49.612841 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:49.612847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:49.612894 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:49.613045 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:49.613736 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.615171 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.616702 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.617441 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.617580 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:49.621175 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:49.621180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:49.621181 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:49.621196 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:49.623532 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:49.624096 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:49.624100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:49.624100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:49.624255 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:49.624571 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:49.624571 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:49.624572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:49.624884 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:49.624965 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:49.625316 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 08:26:49.625468 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":87,"threads_started":1}
I20260619 08:26:49.625746 9778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:49.609787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:46509)
0619 08:26:49.612303 (+ 2516us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:49.612357 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:49.612627 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:49.613082 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:49.613086 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:49.613394 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:49.613596 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:49.613602 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.615320 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.615321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:49.616601 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:49.616604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.617554 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.617555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:49.617700 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:49.618419 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:49.618429 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:49.621033 (+ 2604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:49.623637 (+ 2604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:49.623647 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:49.623650 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:49.623980 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:49.624329 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:49.624329 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:49.624330 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:49.624474 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:49.624955 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:49.624961 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:49.625126 (+ 165us) client_negotiation.cc:769] Sending connection context
0619 08:26:49.625230 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 08:26:49.625419 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2465,"thread_start_us":107,"threads_started":1}
I20260619 08:26:49.626873 9776 heartbeater.cc:347] Connected to a master server at 127.25.127.190:46391
I20260619 08:26:49.626981 9776 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:49.627169 9776 heartbeater.cc:511] Master 127.25.127.190:46391 requested a full tablet report, sending...
I20260619 08:26:49.627944 9586 ts_manager.cc:194] Registered new tserver with Master: 982f047f6beb412c87a89d625f6e5292 (127.25.127.129:33843)
I20260619 08:26:49.629295 9586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:46509
I20260619 08:26:49.630370 9776 heartbeater.cc:503] Master 127.25.127.190:46391 was elected leader, sending a full tablet report...
Jun 19 08:26:49 dist-test-slave-5rhg krb5kdc[9000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857609, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for kudu/127.25.127.129@KRBTEST.COM
I20260619 08:26:49.638533 9782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:49.628669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:33843 (local address 127.0.0.1:57690)
0619 08:26:49.629122 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:49.629161 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:49.629228 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:49.629623 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:49.629629 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:49.629979 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:49.630042 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:49.630044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.631207 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.631208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:49.631986 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:49.631989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.632670 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.632671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:49.632743 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:49.633331 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:49.633337 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:49.635047 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:49.637147 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:49.637156 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:49.637158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:49.637401 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:49.637674 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:49.637675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:49.637676 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:49.637752 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:49.638090 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:49.638097 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:49.638157 (+ 60us) client_negotiation.cc:769] Sending connection context
0619 08:26:49.638271 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 08:26:49.638420 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":93,"threads_started":1}
I20260619 08:26:49.638546 9783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:49.628925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57690 (local address 127.25.127.129:33843)
0619 08:26:49.629372 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:49.629378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:49.629402 (+ 24us) server_negotiation.cc:407] Connection header received
0619 08:26:49.629450 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:49.629452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:49.629470 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:49.629541 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:49.630137 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.631115 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.632078 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.632542 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.632607 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:49.635157 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:49.635168 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:49.635168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:49.635176 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:49.637021 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:49.637516 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:49.637522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:49.637522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:49.637593 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:49.637824 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:49.637825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:49.637826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:49.638004 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:49.638112 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:49.638307 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 08:26:49.638425 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":91,"threads_started":1}
I20260619 08:26:49.639204 9691 tablet_service.cc:1514] Processing DeleteTablet for tablet ff4350c8efb74c05b86a834afee45266 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:26:29 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:57690
I20260619 08:26:49.639940 9784 tablet_replica.cc:330] stopping tablet replica
I20260619 08:26:49.640071 9784 raft_consensus.cc:2229] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:26:49.640180 9784 raft_consensus.cc:2258] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:26:49.640494 9784 ts_tablet_manager.cc:1905] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:26:49.642864 9784 ts_tablet_manager.cc:1918] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:26:49.642973 9784 log.cc:1196] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/wal/wals/ff4350c8efb74c05b86a834afee45266
I20260619 08:26:49.643254 9784 ts_tablet_manager.cc:1939] T ff4350c8efb74c05b86a834afee45266 P 982f047f6beb412c87a89d625f6e5292: Deleting consensus metadata
I20260619 08:26:49.643887 9572 catalog_manager.cc:4841] TS 982f047f6beb412c87a89d625f6e5292 (127.25.127.129:33843): tablet ff4350c8efb74c05b86a834afee45266 (table test-table [id=c82873562a8f4873a8bd41791eb8ac3a]) successfully deleted
W20260619 08:26:49.701610 9781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:49.701835 9781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:49.701861 9781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:49.703020 9781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:49.703065 9781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:49.703094 9781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:26:49.704537 9781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:33433
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=40389
--tserver_master_addrs=127.25.127.190:46391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:49.704743 9781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:49.704926 9781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:49.710124 9790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:49.710201 9789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:49.710150 9792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:49.710413 9781 server_base.cc:1034] running on GCE node
I20260619 08:26:49.710657 9781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:49.710942 9781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:49.712126 9781 hybrid_clock.cc:648] HybridClock initialized: now 1781857609712110 us; error 28 us; skew 500 ppm
Jun 19 08:26:49 dist-test-slave-5rhg krb5kdc[9000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857609, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:49.714795 9781 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:26:49.715282 9781 webserver.cc:460] Webserver started at http://127.25.127.130:40389/ using document root <none> and password file <none>
I20260619 08:26:49.715473 9781 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:49.715528 9781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:49.716917 9781 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:26:49.717592 9799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:49.717810 9781 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:49.717926 9781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "974b12b77cec4cc2827d3f40ca922c43"
format_stamp: "Formatted at 2026-06-19 08:26:28 on dist-test-slave-5rhg"
server_key: "226177830e922cf1980d6a6a4a557d4c"
server_key_iv: "57f18755e9f014342387c7159bdd61fe"
server_key_version: "encryptionkey@0"
I20260619 08:26:49.718286 9781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:49.727862 9781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:49.729812 9781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:49.729933 9781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:49.730127 9781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:49.730448 9781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:26:49.730481 9781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:49.730526 9781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:26:49.730552 9781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:49.736692 9781 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:33433
I20260619 08:26:49.736764 9912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:33433 every 8 connection(s)
I20260619 08:26:49.737133 9781 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:26:49.737448 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 9781
I20260619 08:26:49.737737 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:45615
--local_ip_for_outbound_sockets=127.25.127.131
--tserver_master_addrs=127.25.127.190:46391
--webserver_port=35069
--webserver_interface=127.25.127.131
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 08:26:49 dist-test-slave-5rhg krb5kdc[9000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857609, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:49.747731 9779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:49.737911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:47047 (local address 127.25.127.190:46391)
0619 08:26:49.737983 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:49.737985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:49.738248 (+ 263us) server_negotiation.cc:407] Connection header received
0619 08:26:49.738752 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:49.738754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:49.738771 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:49.738850 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:49.739903 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.740483 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.741524 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.741739 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.741833 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:49.744544 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:49.744545 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:49.744546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:49.744561 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:49.746137 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:49.746633 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:49.746635 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:49.746635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:49.746701 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:49.746998 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:49.746999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:49.747000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:49.747134 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:49.747207 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:49.747516 (+ 309us) server_negotiation.cc:299] Negotiation successful
0619 08:26:49.747632 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 08:26:49.747939 9915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:49.737918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:47047)
0619 08:26:49.738108 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:49.738152 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:49.738655 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:49.738928 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:49.738933 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:49.739546 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:49.739763 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:49.739767 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.740584 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.740585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:49.741423 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:49.741425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.741861 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.741862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:49.741996 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:49.742612 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:49.742622 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:49.744441 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:49.746258 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:49.746267 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:49.746269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:49.746542 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:49.746771 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:49.746772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:49.746772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:49.746916 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:49.747210 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:49.747218 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:49.747372 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 08:26:49.747494 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 08:26:49.747680 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
I20260619 08:26:49.748504 9913 heartbeater.cc:347] Connected to a master server at 127.25.127.190:46391
I20260619 08:26:49.748624 9913 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:49.748826 9913 heartbeater.cc:511] Master 127.25.127.190:46391 requested a full tablet report, sending...
I20260619 08:26:49.749218 9586 ts_manager.cc:194] Registered new tserver with Master: 974b12b77cec4cc2827d3f40ca922c43 (127.25.127.130:33433)
I20260619 08:26:49.749629 9586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:47047
W20260619 08:26:49.823956 9916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:26:49.824348 9916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:26:49.824412 9916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:26:49.825696 9916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:26:49.825752 9916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:26:49.825781 9916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:26:49.827379 9916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:45615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=35069
--tserver_master_addrs=127.25.127.190:46391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:26:49.827687 9916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:26:49.827960 9916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:26:49.833258 9922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:49.833263 9925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:26:49.833391 9923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:26:49.833263 9916 server_base.cc:1034] running on GCE node
I20260619 08:26:49.833662 9916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:26:49.833940 9916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:26:49.835127 9916 hybrid_clock.cc:648] HybridClock initialized: now 1781857609835118 us; error 38 us; skew 500 ppm
Jun 19 08:26:49 dist-test-slave-5rhg krb5kdc[9000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857609, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:26:49.837785 9916 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:26:49.838308 9916 webserver.cc:460] Webserver started at http://127.25.127.131:35069/ using document root <none> and password file <none>
I20260619 08:26:49.838560 9916 fs_manager.cc:359] Metadata directory not provided
I20260619 08:26:49.838658 9916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:26:49.840085 9916 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:49.840745 9932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:49.840927 9916 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:49.841063 9916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "d8399394bb6f482db4248d9b9c51fa49"
format_stamp: "Formatted at 2026-06-19 08:26:29 on dist-test-slave-5rhg"
server_key: "d162f45ec8b3eb89f1ea25d52bec2012"
server_key_iv: "169a135f1215113c001efa49678593c7"
server_key_version: "encryptionkey@0"
I20260619 08:26:49.841447 9916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:26:49.853102 9916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:26:49.855656 9916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:26:49.855825 9916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:26:49.856107 9916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:26:49.856498 9916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:26:49.856559 9916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:49.856604 9916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:26:49.856635 9916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:49.862862 9916 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:45615
I20260619 08:26:49.862906 10046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:45615 every 8 connection(s)
I20260619 08:26:49.863278 9916 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:26:49 dist-test-slave-5rhg krb5kdc[9000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857609, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:26:49.872776 9779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:49.864112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:55431 (local address 127.25.127.190:46391)
0619 08:26:49.864228 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:49.864230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:49.864400 (+ 170us) server_negotiation.cc:407] Connection header received
0619 08:26:49.864949 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:49.864950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:49.864961 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:49.865022 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:49.866019 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.866545 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.867564 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.867776 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.867841 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:49.870163 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:49.870165 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:49.870165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:49.870172 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:49.871425 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:49.871805 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:49.871806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:49.871806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:49.871855 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:49.872141 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:49.872195 (+ 54us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:49.872196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:49.872311 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:49.872366 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:49.872633 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 08:26:49.872705 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 08:26:49.872987 10049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:49.864079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:55431)
0619 08:26:49.864274 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:49.864312 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:49.864812 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:49.865123 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:49.865127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:49.865712 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:26:49.865923 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:49.865927 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.866634 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.866635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:49.867477 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:49.867479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.867858 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.867859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:49.867965 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:49.868627 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:26:49.868636 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:26:49.870068 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:26:49.871503 (+ 1435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:49.871512 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:49.871514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:49.871740 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:49.871912 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:26:49.871912 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:26:49.871912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:26:49.872019 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:26:49.872371 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:26:49.872376 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:26:49.872510 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 08:26:49.872601 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 08:26:49.872732 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":125,"threads_started":1}
I20260619 08:26:49.873311 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 9916
I20260619 08:26:49.873659 10047 heartbeater.cc:347] Connected to a master server at 127.25.127.190:46391
I20260619 08:26:49.873785 10047 heartbeater.cc:464] Registering TS with master...
I20260619 08:26:49.874030 10047 heartbeater.cc:511] Master 127.25.127.190:46391 requested a full tablet report, sending...
I20260619 08:26:49.874436 9586 ts_manager.cc:194] Registered new tserver with Master: d8399394bb6f482db4248d9b9c51fa49 (127.25.127.131:45615)
I20260619 08:26:49.874816 9586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:55431
I20260619 08:26:49.880379 9779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:49.873728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60246 (local address 127.25.127.190:46391)
0619 08:26:49.873771 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:49.873773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:49.873972 (+ 199us) server_negotiation.cc:407] Connection header received
0619 08:26:49.874034 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:49.874035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:49.874043 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:49.874114 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:49.874858 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.875357 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.876315 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.876513 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.876597 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:49.877947 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:49.877948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:49.877949 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:49.877956 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:49.879167 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:49.879539 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:49.879541 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:49.879541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:49.879604 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:49.879815 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:49.879816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:49.879817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:49.879934 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:49.880131 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:49.880198 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 08:26:49.880276 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:26:49.881206 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:26:49.884194 9779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:49.881589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60252 (local address 127.25.127.190:46391)
0619 08:26:49.881665 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:49.881667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:49.881767 (+ 100us) server_negotiation.cc:407] Connection header received
0619 08:26:49.881848 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:49.881849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:49.881897 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:49.881972 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:26:49.882130 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.882607 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.883369 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.883584 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.883670 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:49.883840 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:26:49.883950 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:26:49.884047 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:49.884070 (+ 23us) server_negotiation.cc:299] Negotiation successful
0619 08:26:49.884089 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 08:26:49.884509 9586 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:60252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:26:49.886304 9586 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:26:49.888641 9691 tablet_service.cc:1467] Processing CreateTablet for tablet 7cbc91280ba34652bd4f6cd8466b3d43 (DEFAULT_TABLE table=test-table [id=a7ff93f0b6234eaba05e073ada5eb850]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:26:49.888849 9691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cbc91280ba34652bd4f6cd8466b3d43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:49.890328 9666 tablet_bootstrap.cc:492] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292: Bootstrap starting.
I20260619 08:26:49.890825 9666 tablet_bootstrap.cc:654] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:49.891525 9666 tablet_bootstrap.cc:492] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292: No bootstrap required, opened a new log
I20260619 08:26:49.891573 9666 ts_tablet_manager.cc:1397] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:26:49.891705 9666 raft_consensus.cc:348] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } }
I20260619 08:26:49.891769 9666 raft_consensus.cc:374] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:49.891817 9666 raft_consensus.cc:729] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982f047f6beb412c87a89d625f6e5292, State: Initialized, Role: FOLLOWER
I20260619 08:26:49.891868 9666 consensus_queue.cc:260] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } }
I20260619 08:26:49.891954 9666 ts_tablet_manager.cc:1428] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:26:49.892674 9782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:49.888352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:45615 (local address 127.0.0.1:39946)
0619 08:26:49.888411 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:49.888423 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:49.888476 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:49.888931 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:49.888932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:49.888934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:49.888993 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:49.888995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.889976 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.889977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:49.891149 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:49.891153 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.892346 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.892347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:49.892443 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:49.892465 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 08:26:49.892548 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 08:26:49.892607 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 08:26:49.892729 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:49.888470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39946 (local address 127.25.127.131:45615)
0619 08:26:49.888748 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:49.888770 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:49.888780 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:26:49.888820 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:49.888822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:49.888839 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:49.888911 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:49.889051 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.889883 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.891243 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.892243 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.892347 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:49.892411 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:49.892598 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 08:26:49.892641 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":111,"threads_started":1}
I20260619 08:26:49.893385 9980 tablet_service.cc:1467] Processing CreateTablet for tablet 7cbc91280ba34652bd4f6cd8466b3d43 (DEFAULT_TABLE table=test-table [id=a7ff93f0b6234eaba05e073ada5eb850]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:26:49.893729 9980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cbc91280ba34652bd4f6cd8466b3d43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:49.896682 10056 tablet_bootstrap.cc:492] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: Bootstrap starting.
I20260619 08:26:49.897291 10056 tablet_bootstrap.cc:654] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:49.897807 10056 log.cc:824] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:49.898382 10052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:49.888474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:33433 (local address 127.0.0.1:46086)
0619 08:26:49.888594 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:49.888603 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:49.888641 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:49.891605 (+ 2964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:49.891606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:49.891609 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:49.891660 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:49.891661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.896497 (+ 4836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.896498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:49.897320 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:49.897324 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.898102 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.898104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:49.898177 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:49.898183 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 08:26:49.898244 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 08:26:49.898295 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260619 08:26:49.898665 10054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:49.890405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46086 (local address 127.25.127.130:33433)
0619 08:26:49.891355 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:49.891361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:49.891372 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:26:49.891415 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:49.891419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:49.891498 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:49.891596 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:49.891723 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.896406 (+ 4683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.897441 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:49.898032 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:49.898107 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:49.898172 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:49.898544 (+ 372us) server_negotiation.cc:299] Negotiation successful
0619 08:26:49.898604 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":870,"thread_start_us":97,"threads_started":1}
I20260619 08:26:49.898836 10056 tablet_bootstrap.cc:492] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: No bootstrap required, opened a new log
I20260619 08:26:49.898912 10056 ts_tablet_manager.cc:1397] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:26:49.899278 9847 tablet_service.cc:1467] Processing CreateTablet for tablet 7cbc91280ba34652bd4f6cd8466b3d43 (DEFAULT_TABLE table=test-table [id=a7ff93f0b6234eaba05e073ada5eb850]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:26:49.899605 9847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cbc91280ba34652bd4f6cd8466b3d43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:26:49.900516 10056 raft_consensus.cc:348] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } }
I20260619 08:26:49.900656 10056 raft_consensus.cc:374] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:49.900678 10056 raft_consensus.cc:729] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8399394bb6f482db4248d9b9c51fa49, State: Initialized, Role: FOLLOWER
I20260619 08:26:49.900830 10056 consensus_queue.cc:260] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } }
I20260619 08:26:49.901124 10056 ts_tablet_manager.cc:1428] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:26:49.901162 10047 heartbeater.cc:503] Master 127.25.127.190:46391 was elected leader, sending a full tablet report...
I20260619 08:26:49.902858 10059 tablet_bootstrap.cc:492] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43: Bootstrap starting.
I20260619 08:26:49.903440 10059 tablet_bootstrap.cc:654] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43: Neither blocks nor log segments found. Creating new log.
I20260619 08:26:49.903967 10059 log.cc:824] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43: Log is configured to *not* fsync() on all Append() calls
I20260619 08:26:49.904759 10059 tablet_bootstrap.cc:492] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43: No bootstrap required, opened a new log
I20260619 08:26:49.904845 10059 ts_tablet_manager.cc:1397] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 08:26:49.906418 10059 raft_consensus.cc:348] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } }
I20260619 08:26:49.906607 10059 raft_consensus.cc:374] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:26:49.906679 10059 raft_consensus.cc:729] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 974b12b77cec4cc2827d3f40ca922c43, State: Initialized, Role: FOLLOWER
I20260619 08:26:49.906852 10059 consensus_queue.cc:260] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } }
I20260619 08:26:49.907160 10059 ts_tablet_manager.cc:1428] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 08:26:49.907202 9913 heartbeater.cc:503] Master 127.25.127.190:46391 was elected leader, sending a full tablet report...
W20260619 08:26:49.987807 9914 tablet.cc:2369] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:26:50.113021 9777 tablet.cc:2369] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:26:50.113897 10048 tablet.cc:2369] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:26:50.297021 10058 raft_consensus.cc:482] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:26:50.297158 10058 raft_consensus.cc:504] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } }
I20260619 08:26:50.297621 10058 leader_election.cc:288] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 974b12b77cec4cc2827d3f40ca922c43 (127.25.127.130:33433), 982f047f6beb412c87a89d625f6e5292 (127.25.127.129:33843)
I20260619 08:26:50.300752 10049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:50.297872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:33433 (local address 127.25.127.131:46725)
0619 08:26:50.297943 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:50.297955 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:50.298004 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:50.298156 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:50.298156 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:50.298158 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:50.298253 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:50.298255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:50.299034 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:50.299034 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:50.299833 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:50.299835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:50.300483 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:50.300484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:50.300569 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:50.300573 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:26:50.300628 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 08:26:50.300666 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 08:26:50.300801 10054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:50.297928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:46725 (local address 127.25.127.130:33433)
0619 08:26:50.298004 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:50.298006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:50.298018 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:26:50.298060 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:50.298061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:50.298072 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:50.298132 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:50.298360 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:50.298949 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:50.299930 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:50.300390 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:50.300453 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:50.300460 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:50.300663 (+ 203us) server_negotiation.cc:299] Negotiation successful
0619 08:26:50.300710 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 08:26:50.300982 10062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:50.297892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:33843 (local address 127.25.127.131:49875)
0619 08:26:50.298059 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:50.298069 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:26:50.298117 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:26:50.298360 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:26:50.298361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:26:50.298364 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:26:50.298456 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:50.298457 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:50.299140 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:50.299141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:50.299866 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:26:50.299867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:26:50.300644 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:26:50.300645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:26:50.300723 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:50.300728 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:26:50.300810 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 08:26:50.300854 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260619 08:26:50.300980 10063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:50.297982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:49875 (local address 127.25.127.129:33843)
0619 08:26:50.298097 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:50.298099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:50.298108 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:26:50.298174 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:50.298174 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:50.298206 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:50.298302 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:26:50.298549 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:50.299057 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:50.300033 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:50.300514 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:50.300591 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:50.300677 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:50.300835 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 08:26:50.300890 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":62,"threads_started":1}
I20260619 08:26:50.301432 9867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cbc91280ba34652bd4f6cd8466b3d43" candidate_uuid: "d8399394bb6f482db4248d9b9c51fa49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "974b12b77cec4cc2827d3f40ca922c43" is_pre_election: true
I20260619 08:26:50.301390 9730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cbc91280ba34652bd4f6cd8466b3d43" candidate_uuid: "d8399394bb6f482db4248d9b9c51fa49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982f047f6beb412c87a89d625f6e5292" is_pre_election: true
I20260619 08:26:50.301574 9867 raft_consensus.cc:2454] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8399394bb6f482db4248d9b9c51fa49 in term 0.
I20260619 08:26:50.301574 9730 raft_consensus.cc:2454] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8399394bb6f482db4248d9b9c51fa49 in term 0.
I20260619 08:26:50.301815 9935 leader_election.cc:302] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 982f047f6beb412c87a89d625f6e5292, d8399394bb6f482db4248d9b9c51fa49; no voters:
I20260619 08:26:50.302019 10058 raft_consensus.cc:2781] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:26:50.302084 10058 raft_consensus.cc:482] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:26:50.302101 10058 raft_consensus.cc:3037] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:50.302871 10058 raft_consensus.cc:504] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } }
I20260619 08:26:50.303089 10058 leader_election.cc:288] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [CANDIDATE]: Term 1 election: Requested vote from peers 974b12b77cec4cc2827d3f40ca922c43 (127.25.127.130:33433), 982f047f6beb412c87a89d625f6e5292 (127.25.127.129:33843)
I20260619 08:26:50.303194 9730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cbc91280ba34652bd4f6cd8466b3d43" candidate_uuid: "d8399394bb6f482db4248d9b9c51fa49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982f047f6beb412c87a89d625f6e5292"
I20260619 08:26:50.303290 9730 raft_consensus.cc:3037] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:50.303293 9867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cbc91280ba34652bd4f6cd8466b3d43" candidate_uuid: "d8399394bb6f482db4248d9b9c51fa49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "974b12b77cec4cc2827d3f40ca922c43"
I20260619 08:26:50.303375 9867 raft_consensus.cc:3037] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:26:50.304060 9730 raft_consensus.cc:2454] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8399394bb6f482db4248d9b9c51fa49 in term 1.
I20260619 08:26:50.304147 9867 raft_consensus.cc:2454] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8399394bb6f482db4248d9b9c51fa49 in term 1.
I20260619 08:26:50.304301 9935 leader_election.cc:302] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 982f047f6beb412c87a89d625f6e5292, d8399394bb6f482db4248d9b9c51fa49; no voters:
I20260619 08:26:50.304459 10058 raft_consensus.cc:2781] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:26:50.304622 10058 raft_consensus.cc:686] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [term 1 LEADER]: Becoming Leader. State: Replica: d8399394bb6f482db4248d9b9c51fa49, State: Running, Role: LEADER
I20260619 08:26:50.304755 10058 consensus_queue.cc:237] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } }
I20260619 08:26:50.305691 9586 catalog_manager.cc:5495] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 reported cstate change: term changed from 0 to 1, leader changed from <none> to d8399394bb6f482db4248d9b9c51fa49 (127.25.127.131). New cstate: current_term: 1 leader_uuid: "d8399394bb6f482db4248d9b9c51fa49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } health_report { overall_health: UNKNOWN } } }
I20260619 08:26:50.334440 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:50.332244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39956 (local address 127.25.127.131:45615)
0619 08:26:50.332302 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:50.332304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:50.332313 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:26:50.332360 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:50.332361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:50.332380 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:50.332442 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:26:50.332568 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:50.333045 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:50.333740 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:50.333929 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:50.333980 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:50.334181 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:26:50.334231 (+ 50us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 08:26:50.334361 (+ 130us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39956: token expired
Metrics: {"server-negotiator.queue_time_us":33}
W20260619 08:26:50.334574 10053 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39956: token expired
I20260619 08:26:50.340451 9779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:50.334883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60254 (local address 127.25.127.190:46391)
0619 08:26:50.334943 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:50.334945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:50.334956 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:26:50.335011 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:50.335011 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:50.335020 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:50.335063 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:26:50.335526 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:50.335982 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:50.336678 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:50.336857 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:50.336905 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:50.338207 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:26:50.338208 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:26:50.338209 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:26:50.338215 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:26:50.339475 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:50.339796 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:50.339797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:50.339797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:50.339845 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:26:50.340031 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:26:50.340031 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:26:50.340032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:26:50.340128 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:26:50.340215 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:50.340321 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 08:26:50.340387 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 08:26:50.343551 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:50.341464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39968 (local address 127.25.127.131:45615)
0619 08:26:50.341527 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:26:50.341529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:26:50.341544 (+ 15us) server_negotiation.cc:407] Connection header received
0619 08:26:50.341564 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:26:50.341565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:26:50.341575 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:26:50.341619 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:26:50.341702 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:50.342141 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:50.342877 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:26:50.343053 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:26:50.343100 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:26:50.343231 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:26:50.343322 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:26:50.343361 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 08:26:50.343417 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 08:26:50.343434 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:26:50.345515 9730 raft_consensus.cc:1261] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Refusing update from remote peer d8399394bb6f482db4248d9b9c51fa49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:26:50.345527 9867 raft_consensus.cc:1261] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43 [term 1 FOLLOWER]: Refusing update from remote peer d8399394bb6f482db4248d9b9c51fa49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:26:50.345839 10058 consensus_queue.cc:1035] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:26:50.345989 10064 consensus_queue.cc:1035] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:26:50.348618 10068 mvcc.cc:205] Tried to move back new op lower bound from 7298488771972100096 to 7298488771808940032. Current Snapshot: MvccSnapshot[applied={T|T < 7298488771972100096 or (T in {7298488771972100096})}]
I20260619 08:26:50.351073 9586 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:26:50.351183 9586 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:26:50.351907 9586 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: Sending DeleteTablet for 3 replicas of tablet 7cbc91280ba34652bd4f6cd8466b3d43
I20260619 08:26:50.352298 9847 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cbc91280ba34652bd4f6cd8466b3d43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:26:50 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:46086
I20260619 08:26:50.352311 9980 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cbc91280ba34652bd4f6cd8466b3d43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:26:50 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:39946
I20260619 08:26:50.352535 10076 tablet_replica.cc:330] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: stopping tablet replica
I20260619 08:26:50.352512 9691 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cbc91280ba34652bd4f6cd8466b3d43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:26:50 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:57690
I20260619 08:26:50.352684 10076 raft_consensus.cc:2229] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:26:50.352730 10078 tablet_replica.cc:330] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292: stopping tablet replica
I20260619 08:26:50.352799 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 9624
I20260619 08:26:50.352834 10078 raft_consensus.cc:2229] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:26:50.352888 10078 raft_consensus.cc:2258] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:26:50.352896 10076 raft_consensus.cc:2258] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:26:50.353716 10076 ts_tablet_manager.cc:1905] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:26:50.355849 10076 ts_tablet_manager.cc:1918] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:26:50.355957 10076 log.cc:1196] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/7cbc91280ba34652bd4f6cd8466b3d43
I20260619 08:26:50.356290 10076 ts_tablet_manager.cc:1939] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: Deleting consensus metadata
I20260619 08:26:50.356776 9573 catalog_manager.cc:4841] TS d8399394bb6f482db4248d9b9c51fa49 (127.25.127.131:45615): tablet 7cbc91280ba34652bd4f6cd8466b3d43 (table test-table [id=a7ff93f0b6234eaba05e073ada5eb850]) successfully deleted
W20260619 08:26:50.359663 9572 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.127.129:33843 (error 108)
I20260619 08:26:50.360059 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 9781
I20260619 08:26:50.360221 10052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:50.360067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:33843 (local address 127.0.0.1:57694)
0619 08:26:50.360127 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:50.360187 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.129:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 08:26:50.360222 10077 tablet_replica.cc:330] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43: stopping tablet replica
W20260619 08:26:50.360339 9572 catalog_manager.cc:4584] TS 982f047f6beb412c87a89d625f6e5292 (127.25.127.129:33843): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7cbc91280ba34652bd4f6cd8466b3d43: Network error: Client connection negotiation failed: client connection to 127.25.127.129:33843: connect: Connection refused (error 111)
I20260619 08:26:50.365082 10052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:26:50.364953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:33433 (local address 127.0.0.1:46102)
0619 08:26:50.365010 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 08:26:50.365043 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.130:33433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 08:26:50.365172 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 9916
I20260619 08:26:50.369797 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 9552
I20260619 08:27:10.375699 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:46391
--webserver_interface=127.25.127.190
--webserver_port=42677
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:46391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:10.459209 10080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:10.459420 10080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:10.459443 10080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:10.460755 10080 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 08:27:10.460801 10080 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 08:27:10.460815 10080 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 08:27:10.460826 10080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:10.460840 10080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:10.460883 10080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:10.460909 10080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:10.460925 10080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:10.462422 10080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:46391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:46391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=42677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:10.462707 10080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:10.462946 10080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:10.468267 10085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:10.468250 10086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:10.468277 10080 server_base.cc:1034] running on GCE node
W20260619 08:27:10.468250 10088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:10.468710 10080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:10.469007 10080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:10.470211 10080 hybrid_clock.cc:648] HybridClock initialized: now 1781857630470203 us; error 49 us; skew 500 ppm
Jun 19 08:27:10 dist-test-slave-5rhg krb5kdc[9000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857630, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:10.472755 10080 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:27:10.473311 10080 webserver.cc:460] Webserver started at http://127.25.127.190:42677/ using document root <none> and password file <none>
I20260619 08:27:10.473565 10080 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:10.473641 10080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:10.475023 10080 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:10.475760 10095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:10.475972 10080 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:10.476089 10080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "5bc56cd048ef4b628769f12a41761435"
format_stamp: "Formatted at 2026-06-19 08:26:28 on dist-test-slave-5rhg"
server_key: "b8bd1e93f9b109296eab09dcb08de45a"
server_key_iv: "14f98c9283c72798181ded1463335786"
server_key_version: "encryptionkey@0"
I20260619 08:27:10.476495 10080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:10.496762 10080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:10.498654 10080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:10.498828 10080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:10.503337 10080 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:46391
I20260619 08:27:10.503417 10148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:46391 every 8 connection(s)
I20260619 08:27:10.503834 10080 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:10.505373 10149 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:27:10.506100 10149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: Bootstrap starting.
I20260619 08:27:10.510774 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 10080
I20260619 08:27:10.511065 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:33843
--local_ip_for_outbound_sockets=127.25.127.129
--tserver_master_addrs=127.25.127.190:46391
--webserver_port=37909
--webserver_interface=127.25.127.129
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:27:10.515209 10149 log.cc:824] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:10.517560 10149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:27:10.517879 10149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: Bootstrap complete.
I20260619 08:27:10.520224 10149 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc56cd048ef4b628769f12a41761435" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46391 } }
I20260619 08:27:10.520509 10149 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bc56cd048ef4b628769f12a41761435, State: Initialized, Role: FOLLOWER
I20260619 08:27:10.520707 10149 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc56cd048ef4b628769f12a41761435" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46391 } }
I20260619 08:27:10.520792 10149 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:10.520846 10149 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:10.520895 10149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 2 FOLLOWER]: Advancing to term 3
I20260619 08:27:10.521883 10149 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc56cd048ef4b628769f12a41761435" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46391 } }
I20260619 08:27:10.522010 10149 leader_election.cc:302] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bc56cd048ef4b628769f12a41761435; no voters:
I20260619 08:27:10.522238 10149 leader_election.cc:288] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 08:27:10.522274 10153 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 08:27:10.522392 10153 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [term 3 LEADER]: Becoming Leader. State: Replica: 5bc56cd048ef4b628769f12a41761435, State: Running, Role: LEADER
I20260619 08:27:10.522533 10149 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:10.522552 10153 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc56cd048ef4b628769f12a41761435" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46391 } }
I20260619 08:27:10.522854 10153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bc56cd048ef4b628769f12a41761435. Latest consensus state: current_term: 3 leader_uuid: "5bc56cd048ef4b628769f12a41761435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc56cd048ef4b628769f12a41761435" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46391 } } }
I20260619 08:27:10.523044 10154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5bc56cd048ef4b628769f12a41761435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bc56cd048ef4b628769f12a41761435" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46391 } } }
I20260619 08:27:10.523131 10153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:10.523350 10154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:10.523497 10162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:10.524401 10162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:10.524542 10162 catalog_manager.cc:1259] Loaded cluster ID: 4da34608d19742fc9f0b5eb9299aa48b
I20260619 08:27:10.524788 10162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:10.525981 10162 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:10.526225 10162 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: Loaded TSK: 0
I20260619 08:27:10.526304 10162 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5bc56cd048ef4b628769f12a41761435: Loaded TSK: 1
F20260619 08:27:10.526566 10162 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb80fff39ad google::LogMessage::Fail() at ??:0
@ 0x7fb80fff688a google::LogMessage::SendToLog() at ??:0
@ 0x7fb80fff34de google::LogMessage::Flush() at ??:0
@ 0x7fb80fff4a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb812760104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fb81275dd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fb817f4d1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fb817f4d48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fb817f4d594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fb817f13c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fb817f1dad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb811ec10d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb811eb6954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb812d1a6db start_thread at ??:0
@ 0x7fb80eaf161f clone at ??:0
W20260619 08:27:10.612849 10151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:10.613054 10151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:10.613080 10151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:10.614248 10151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:10.614290 10151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:10.614317 10151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:27:10.615840 10151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:33843
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=37909
--tserver_master_addrs=127.25.127.190:46391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:10.616083 10151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:10.616356 10151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:10.621398 10190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:10.621482 10193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:10.621393 10191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:10.621904 10151 server_base.cc:1034] running on GCE node
I20260619 08:27:10.622112 10151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:10.622394 10151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:10.623553 10151 hybrid_clock.cc:648] HybridClock initialized: now 1781857630623538 us; error 30 us; skew 500 ppm
Jun 19 08:27:10 dist-test-slave-5rhg krb5kdc[9000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857630, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:10.626264 10151 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:27:10.626717 10151 webserver.cc:460] Webserver started at http://127.25.127.129:37909/ using document root <none> and password file <none>
I20260619 08:27:10.626952 10151 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:10.627023 10151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:10.628441 10151 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:27:10.629220 10200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:10.629429 10151 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:27:10.629536 10151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "982f047f6beb412c87a89d625f6e5292"
format_stamp: "Formatted at 2026-06-19 08:26:28 on dist-test-slave-5rhg"
server_key: "d04fc56c66e8b751d333e8662287910b"
server_key_iv: "d1fe2fab7ed278c2a5b47dc864a84a84"
server_key_version: "encryptionkey@0"
I20260619 08:27:10.629899 10151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:10.643746 10151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:10.646091 10151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:10.646261 10151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:10.646517 10151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:10.646971 10207 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:27:10.649169 10151 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:27:10.649240 10151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:27:10.649289 10151 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:27:10.649936 10151 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:27:10.649996 10151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 08:27:10.650101 10207 tablet_bootstrap.cc:492] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292: Bootstrap starting.
I20260619 08:27:10.657423 10151 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:33843
I20260619 08:27:10.657496 10315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:33843 every 8 connection(s)
I20260619 08:27:10.657892 10151 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:27:10.662544 10207 log.cc:824] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:10.665531 10207 tablet_bootstrap.cc:492] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:27:10.665861 10207 tablet_bootstrap.cc:492] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292: Bootstrap complete.
I20260619 08:27:10.666285 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 10151
I20260619 08:27:10.666543 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:33433
--local_ip_for_outbound_sockets=127.25.127.130
--tserver_master_addrs=127.25.127.190:46391
--webserver_port=40389
--webserver_interface=127.25.127.130
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:27:10.667197 10207 ts_tablet_manager.cc:1397] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.004s
I20260619 08:27:10.668699 10207 raft_consensus.cc:348] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } }
I20260619 08:27:10.669051 10207 raft_consensus.cc:729] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982f047f6beb412c87a89d625f6e5292, State: Initialized, Role: FOLLOWER
I20260619 08:27:10.669250 10207 consensus_queue.cc:260] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } }
I20260619 08:27:10.669552 10207 ts_tablet_manager.cc:1428] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260619 08:27:10.730553 10318 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:27:10.658706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:57129)
0619 08:27:10.658903 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:10.658943 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:10.659401 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:10.730332 (+ 70931us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":109,"threads_started":1}
W20260619 08:27:10.730782 10316 heartbeater.cc:644] Failed to heartbeat to 127.25.127.190:46391 (0 consecutive failures): Network error: Failed to ping master at 127.25.127.190:46391: Client connection negotiation failed: client connection to 127.25.127.190:46391: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 08:27:10.731161 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:10.731032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:43525)
0619 08:27:10.731086 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:10.731131 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 08:27:10.731575 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:10.731447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:35269)
0619 08:27:10.731521 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:10.731553 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260619 08:27:10.731747 10316 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 08:27:10.752368 10320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:10.752578 10320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:10.752604 10320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:10.753728 10320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:10.753763 10320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:10.753789 10320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:27:10.755179 10320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:33433
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=40389
--tserver_master_addrs=127.25.127.190:46391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:10.755371 10320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:10.755554 10320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:10.760812 10326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:10.760857 10320 server_base.cc:1034] running on GCE node
W20260619 08:27:10.760802 10329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:10.760802 10327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:10.761278 10320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:10.761600 10320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:10.762758 10320 hybrid_clock.cc:648] HybridClock initialized: now 1781857630762740 us; error 48 us; skew 500 ppm
Jun 19 08:27:10 dist-test-slave-5rhg krb5kdc[9000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857630, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:10.765393 10320 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:27:10.765844 10320 webserver.cc:460] Webserver started at http://127.25.127.130:40389/ using document root <none> and password file <none>
I20260619 08:27:10.766085 10320 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:10.766132 10320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:10.767550 10320 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:27:10.768302 10336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:10.768499 10320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:27:10.768576 10320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "974b12b77cec4cc2827d3f40ca922c43"
format_stamp: "Formatted at 2026-06-19 08:26:28 on dist-test-slave-5rhg"
server_key: "226177830e922cf1980d6a6a4a557d4c"
server_key_iv: "57f18755e9f014342387c7159bdd61fe"
server_key_version: "encryptionkey@0"
I20260619 08:27:10.768957 10320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:10.785388 10320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:10.787443 10320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:10.787606 10320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:10.787861 10320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:10.788417 10343 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:27:10.789913 10320 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:27:10.789981 10320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:27:10.790030 10320 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:27:10.790685 10320 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:27:10.790743 10320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 08:27:10.790839 10343 tablet_bootstrap.cc:492] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43: Bootstrap starting.
I20260619 08:27:10.798206 10320 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:33433
I20260619 08:27:10.798317 10451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:33433 every 8 connection(s)
I20260619 08:27:10.798650 10320 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:27:10.799854 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:10.799327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:51715)
0619 08:27:10.799547 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:10.799657 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":74,"threads_started":1}
W20260619 08:27:10.800192 10452 heartbeater.cc:644] Failed to heartbeat to 127.25.127.190:46391 (0 consecutive failures): Network error: Failed to ping master at 127.25.127.190:46391: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
I20260619 08:27:10.800545 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:10.800396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:50621)
0619 08:27:10.800464 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:10.800502 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 08:27:10.800915 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:10.800806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:47429)
0619 08:27:10.800862 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:10.800892 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 08:27:10.801061 10452 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 08:27:10.801275 10343 log.cc:824] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:10.802233 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 10320
I20260619 08:27:10.802531 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:45615
--local_ip_for_outbound_sockets=127.25.127.131
--tserver_master_addrs=127.25.127.190:46391
--webserver_port=35069
--webserver_interface=127.25.127.131
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:27:10.803584 10343 tablet_bootstrap.cc:492] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:27:10.803908 10343 tablet_bootstrap.cc:492] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43: Bootstrap complete.
I20260619 08:27:10.805246 10343 ts_tablet_manager.cc:1397] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.011s
I20260619 08:27:10.807518 10343 raft_consensus.cc:348] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } }
I20260619 08:27:10.807956 10343 raft_consensus.cc:729] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 974b12b77cec4cc2827d3f40ca922c43, State: Initialized, Role: FOLLOWER
I20260619 08:27:10.808137 10343 consensus_queue.cc:260] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } }
I20260619 08:27:10.808466 10343 ts_tablet_manager.cc:1428] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260619 08:27:10.808736 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:10.808623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:44127)
0619 08:27:10.808665 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:10.808704 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 08:27:10.809054 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:10.808950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:57407)
0619 08:27:10.808991 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:10.809027 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260619 08:27:10.885574 10456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:10.885787 10456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:10.885811 10456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:10.887001 10456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:10.887046 10456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:10.887072 10456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:27:10.888573 10456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:45615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=35069
--tserver_master_addrs=127.25.127.190:46391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:10.888831 10456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:10.889072 10456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:10.894093 10463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:10.894122 10465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:10.894104 10462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:10.894368 10456 server_base.cc:1034] running on GCE node
I20260619 08:27:10.894644 10456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:10.894892 10456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:10.896072 10456 hybrid_clock.cc:648] HybridClock initialized: now 1781857630896036 us; error 49 us; skew 500 ppm
Jun 19 08:27:10 dist-test-slave-5rhg krb5kdc[9000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857630, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:10.898655 10456 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:27:10.899153 10456 webserver.cc:460] Webserver started at http://127.25.127.131:35069/ using document root <none> and password file <none>
I20260619 08:27:10.899387 10456 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:10.899435 10456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:10.900856 10456 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:10.901482 10472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:10.901661 10456 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:27:10.901764 10456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "d8399394bb6f482db4248d9b9c51fa49"
format_stamp: "Formatted at 2026-06-19 08:26:29 on dist-test-slave-5rhg"
server_key: "d162f45ec8b3eb89f1ea25d52bec2012"
server_key_iv: "169a135f1215113c001efa49678593c7"
server_key_version: "encryptionkey@0"
I20260619 08:27:10.902143 10456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:10.931906 10456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:10.934535 10456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:10.934720 10456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:10.934988 10456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:10.935336 10456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:10.935395 10456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:10.935441 10456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:10.935465 10456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:10.942111 10456 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:45615
I20260619 08:27:10.942157 10586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:45615 every 8 connection(s)
I20260619 08:27:10.942540 10456 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:27:10.943657 10589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:10.943246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:39233)
0619 08:27:10.943420 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:10.943499 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
W20260619 08:27:10.944087 10587 heartbeater.cc:644] Failed to heartbeat to 127.25.127.190:46391 (0 consecutive failures): Network error: Failed to ping master at 127.25.127.190:46391: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
I20260619 08:27:10.944449 10589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:10.944341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:41219)
0619 08:27:10.944398 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:10.944429 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 08:27:10.944821 10589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:10.944718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:51285)
0619 08:27:10.944762 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:10.944786 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 08:27:10.944984 10587 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 08:27:10.948107 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 10456
I20260619 08:27:11.732393 10591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:11.732124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:39063)
0619 08:27:11.732295 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:11.732332 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":110,"threads_started":1}
I20260619 08:27:11.809645 10592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:11.809436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:54175)
0619 08:27:11.809559 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:11.809595 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":56,"threads_started":1}
I20260619 08:27:11.830487 10593 raft_consensus.cc:482] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:27:11.830607 10593 raft_consensus.cc:504] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } }
I20260619 08:27:11.830972 10593 leader_election.cc:288] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d8399394bb6f482db4248d9b9c51fa49 (127.25.127.131:45615), 974b12b77cec4cc2827d3f40ca922c43 (127.25.127.130:33433)
Jun 19 08:27:11 dist-test-slave-5rhg krb5kdc[9000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857630, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.131@KRBTEST.COM
Jun 19 08:27:11 dist-test-slave-5rhg krb5kdc[9000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857630, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.130@KRBTEST.COM
I20260619 08:27:11.841521 10594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:11.831257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:57257 (local address 127.25.127.131:45615)
0619 08:27:11.831430 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:11.831442 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:11.831471 (+ 29us) server_negotiation.cc:407] Connection header received
0619 08:27:11.832027 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:11.832030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:11.832096 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:11.832254 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:11.832842 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:11.833857 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:11.834806 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:11.835213 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:11.835731 (+ 518us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:11.837810 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:11.837831 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:11.837832 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:11.837844 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:11.839821 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:11.840250 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:11.840253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:11.840254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:11.840350 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:11.840655 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:11.840656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:11.840656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:11.840894 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:11.840982 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:11.841286 (+ 304us) server_negotiation.cc:299] Negotiation successful
0619 08:27:11.841417 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
I20260619 08:27:11.841535 10591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:11.831155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:45615 (local address 127.25.127.129:57257)
0619 08:27:11.831209 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:11.831221 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:11.831277 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:11.832244 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:11.832245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:11.832455 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:11.832672 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:11.832678 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:11.833953 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:11.833954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:11.834712 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:11.834714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:11.835354 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:11.835356 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:11.835440 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:11.836195 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:27:11.836200 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:11.837705 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:11.839900 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:11.839906 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:11.839907 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:11.840118 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:11.840430 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:11.840431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:11.840431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:11.840573 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:11.840976 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:11.840983 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:11.841119 (+ 136us) client_negotiation.cc:769] Sending connection context
0619 08:27:11.841223 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 08:27:11.841412 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":47}
I20260619 08:27:11.841827 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:11.831310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:33433 (local address 127.25.127.129:36337)
0619 08:27:11.831405 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:11.831415 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:11.831457 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:11.831887 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:11.831890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:11.832422 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:11.832672 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:11.832678 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:11.833839 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:11.833840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:11.834712 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:11.834715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:11.835322 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:11.835322 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:11.835432 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:11.836100 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:27:11.836108 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:11.838067 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:11.840112 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:11.840113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:11.840113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:11.840318 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:11.840583 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:11.840584 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:11.840584 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:11.840648 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:11.841036 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:11.841037 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:11.841123 (+ 86us) client_negotiation.cc:769] Sending connection context
0619 08:27:11.841218 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 08:27:11.841491 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":188,"thread_start_us":49,"threads_started":1}
I20260619 08:27:11.841563 10596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:11.831388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:36337 (local address 127.25.127.130:33433)
0619 08:27:11.831512 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:11.831520 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:11.831557 (+ 37us) server_negotiation.cc:407] Connection header received
0619 08:27:11.831644 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:11.831646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:11.831685 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:11.831821 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:11.832794 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:11.833744 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:11.834815 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:11.835215 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:11.835337 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:11.838189 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:11.838192 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:11.838192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:11.838201 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:11.840028 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:11.840416 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:11.840418 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:11.840419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:11.840511 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:11.840732 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:11.840733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:11.840733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:11.840956 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:11.841048 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:11.841285 (+ 237us) server_negotiation.cc:299] Negotiation successful
0619 08:27:11.841435 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":55,"threads_started":1}
W20260619 08:27:11.842363 10204 leader_election.cc:341] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d8399394bb6f482db4248d9b9c51fa49 (127.25.127.131:45615): Not found: Tablet not found: 7cbc91280ba34652bd4f6cd8466b3d43
I20260619 08:27:11.842381 10405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cbc91280ba34652bd4f6cd8466b3d43" candidate_uuid: "982f047f6beb412c87a89d625f6e5292" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "974b12b77cec4cc2827d3f40ca922c43" is_pre_election: true
I20260619 08:27:11.842576 10405 raft_consensus.cc:2454] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 982f047f6beb412c87a89d625f6e5292 in term 1.
I20260619 08:27:11.842857 10204 leader_election.cc:302] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 974b12b77cec4cc2827d3f40ca922c43, 982f047f6beb412c87a89d625f6e5292; no voters: d8399394bb6f482db4248d9b9c51fa49
I20260619 08:27:11.842989 10593 raft_consensus.cc:2781] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 08:27:11.843084 10593 raft_consensus.cc:482] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:27:11.843125 10593 raft_consensus.cc:3037] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:27:11.844277 10593 raft_consensus.cc:504] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } }
I20260619 08:27:11.844414 10593 leader_election.cc:288] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [CANDIDATE]: Term 2 election: Requested vote from peers d8399394bb6f482db4248d9b9c51fa49 (127.25.127.131:45615), 974b12b77cec4cc2827d3f40ca922c43 (127.25.127.130:33433)
I20260619 08:27:11.844612 10405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cbc91280ba34652bd4f6cd8466b3d43" candidate_uuid: "982f047f6beb412c87a89d625f6e5292" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "974b12b77cec4cc2827d3f40ca922c43"
I20260619 08:27:11.844712 10405 raft_consensus.cc:3037] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43 [term 1 FOLLOWER]: Advancing to term 2
W20260619 08:27:11.844722 10204 leader_election.cc:341] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d8399394bb6f482db4248d9b9c51fa49 (127.25.127.131:45615): Not found: Tablet not found: 7cbc91280ba34652bd4f6cd8466b3d43
I20260619 08:27:11.845752 10405 raft_consensus.cc:2454] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 982f047f6beb412c87a89d625f6e5292 in term 2.
I20260619 08:27:11.845918 10204 leader_election.cc:302] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 974b12b77cec4cc2827d3f40ca922c43, 982f047f6beb412c87a89d625f6e5292; no voters: d8399394bb6f482db4248d9b9c51fa49
I20260619 08:27:11.846046 10593 raft_consensus.cc:2781] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 08:27:11.846165 10593 raft_consensus.cc:686] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [term 2 LEADER]: Becoming Leader. State: Replica: 982f047f6beb412c87a89d625f6e5292, State: Running, Role: LEADER
I20260619 08:27:11.846272 10593 consensus_queue.cc:237] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } }
I20260619 08:27:11.846473 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:11.846367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:35407)
0619 08:27:11.846414 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:11.846442 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 08:27:11.945566 10599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:11.945298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:44701)
0619 08:27:11.945476 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:11.945526 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":75,"threads_started":1}
I20260619 08:27:12.393754 10405 raft_consensus.cc:1261] T 7cbc91280ba34652bd4f6cd8466b3d43 P 974b12b77cec4cc2827d3f40ca922c43 [term 2 FOLLOWER]: Refusing update from remote peer 982f047f6beb412c87a89d625f6e5292: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260619 08:27:12.394169 10600 consensus_queue.cc:1035] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 [LEADER]: Connected to new peer: Peer: permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 08:27:12.395212 10603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:12.395029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:43783)
0619 08:27:12.395115 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:12.395153 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72,"thread_start_us":41,"threads_started":1}
I20260619 08:27:12.396215 10603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:12.396083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:41497)
0619 08:27:12.396119 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:12.396179 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 08:27:12.396378 10608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:12.396136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:33609)
0619 08:27:12.396294 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:12.396329 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":100,"threads_started":1}
W20260619 08:27:12.396417 10204 consensus_peers.cc:487] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292 -> Peer d8399394bb6f482db4248d9b9c51fa49 (127.25.127.131:45615): Couldn't send request to peer d8399394bb6f482db4248d9b9c51fa49. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7cbc91280ba34652bd4f6cd8466b3d43. This is attempt 1: this message will repeat every 5th retry.
I20260619 08:27:12.852790 10609 ts_tablet_manager.cc:927] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: Initiating tablet copy from peer 982f047f6beb412c87a89d625f6e5292 (127.25.127.129:33843)
I20260619 08:27:12.853034 10609 tablet_copy_client.cc:316] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: tablet copy: Beginning tablet copy session from remote peer at address 127.25.127.129:33843
Jun 19 08:27:12 dist-test-slave-5rhg krb5kdc[9000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857630, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.129@KRBTEST.COM
I20260619 08:27:12.868130 10612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:12.853460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:35361 (local address 127.25.127.129:33843)
0619 08:27:12.853607 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:12.853611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:12.853621 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:27:12.853647 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:12.853649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:12.853660 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:12.853732 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:12.854203 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:12.854921 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:12.855721 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:12.856049 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:12.856108 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:12.865120 (+ 9012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:12.865133 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:12.865133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:12.865141 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:12.866773 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:12.867223 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:12.867227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:12.867228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:12.867295 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:12.867532 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:12.867533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:12.867533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:12.867715 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:12.867788 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:12.867989 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 08:27:12.868061 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":67,"threads_started":1}
I20260619 08:27:12.868127 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:12.853357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:33843 (local address 127.25.127.131:35361)
0619 08:27:12.853527 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:12.853543 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:12.853598 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:12.853743 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:12.853746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:12.854034 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:12.854117 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:12.854119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:12.855033 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:12.855034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:12.855632 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:12.855633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:12.856134 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:12.856135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:12.856272 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:12.863429 (+ 7157us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:27:12.863436 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:12.865004 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:12.866868 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:12.866877 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:12.866879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:12.867124 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:12.867366 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:12.867367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:12.867367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:12.867425 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:12.867804 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:12.867811 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:12.867866 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 08:27:12.867940 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 08:27:12.868044 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":88,"threads_started":1}
I20260619 08:27:12.868759 10290 tablet_copy_service.cc:140] P 982f047f6beb412c87a89d625f6e5292: Received BeginTabletCopySession request for tablet 7cbc91280ba34652bd4f6cd8466b3d43 from peer d8399394bb6f482db4248d9b9c51fa49 ({username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:35361)
I20260619 08:27:12.868827 10290 tablet_copy_service.cc:161] P 982f047f6beb412c87a89d625f6e5292: Beginning new tablet copy session on tablet 7cbc91280ba34652bd4f6cd8466b3d43 from peer d8399394bb6f482db4248d9b9c51fa49 at {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:35361: session id = d8399394bb6f482db4248d9b9c51fa49-7cbc91280ba34652bd4f6cd8466b3d43
I20260619 08:27:12.869372 10290 tablet_copy_source_session.cc:205] T 7cbc91280ba34652bd4f6cd8466b3d43 P 982f047f6beb412c87a89d625f6e5292: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 08:27:12.870441 10609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cbc91280ba34652bd4f6cd8466b3d43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:12.873649 10609 tablet_copy_client.cc:791] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: tablet copy: Starting download of 0 data blocks...
I20260619 08:27:12.873819 10609 tablet_copy_client.cc:655] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: tablet copy: Starting download of 1 WAL segments...
I20260619 08:27:12.874814 10609 tablet_copy_client.cc:531] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 08:27:12.875533 10609 tablet_bootstrap.cc:492] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: Bootstrap starting.
I20260619 08:27:12.877012 10609 log.cc:824] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:12.879431 10609 tablet_bootstrap.cc:492] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:27:12.879678 10609 tablet_bootstrap.cc:492] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: Bootstrap complete.
I20260619 08:27:12.879818 10609 ts_tablet_manager.cc:1397] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 08:27:12.880985 10609 raft_consensus.cc:348] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } }
I20260619 08:27:12.881146 10609 raft_consensus.cc:729] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8399394bb6f482db4248d9b9c51fa49, State: Initialized, Role: FOLLOWER
I20260619 08:27:12.881292 10609 consensus_queue.cc:260] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8399394bb6f482db4248d9b9c51fa49" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45615 } } peers { permanent_uuid: "974b12b77cec4cc2827d3f40ca922c43" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 33433 } } peers { permanent_uuid: "982f047f6beb412c87a89d625f6e5292" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33843 } }
I20260619 08:27:12.881551 10609 ts_tablet_manager.cc:1428] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49: Time spent starting tablet: real 0.002s user 0.000s sys 0.004s
I20260619 08:27:12.881876 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:12.881743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:42517)
0619 08:27:12.881811 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:12.881845 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 08:27:12.881913 10290 tablet_copy_service.cc:342] P 982f047f6beb412c87a89d625f6e5292: Request end of tablet copy session d8399394bb6f482db4248d9b9c51fa49-7cbc91280ba34652bd4f6cd8466b3d43 received from {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:35361
I20260619 08:27:12.882014 10290 tablet_copy_service.cc:433] P 982f047f6beb412c87a89d625f6e5292: ending tablet copy session d8399394bb6f482db4248d9b9c51fa49-7cbc91280ba34652bd4f6cd8466b3d43 on tablet 7cbc91280ba34652bd4f6cd8466b3d43 with peer d8399394bb6f482db4248d9b9c51fa49
I20260619 08:27:12.882254 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:12.882151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:58945)
0619 08:27:12.882201 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:12.882230 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 08:27:13.308187 10540 raft_consensus.cc:1203] T 7cbc91280ba34652bd4f6cd8466b3d43 P d8399394bb6f482db4248d9b9c51fa49 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260619 08:27:13.308532 10616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:13.308050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:52137)
0619 08:27:13.308429 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:13.308476 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":51,"threads_started":1}
I20260619 08:27:13.308701 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:13.308572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:56947)
0619 08:27:13.308630 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:13.308667 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 08:27:13.396914 10617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:13.396699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:40863)
0619 08:27:13.396816 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:13.396858 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260619 08:27:14.309361 10618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:14.309105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:42333)
0619 08:27:14.309264 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:14.309305 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":76,"threads_started":1}
I20260619 08:27:14.309361 10619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:14.309107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:58023)
0619 08:27:14.309263 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:14.309305 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":77,"threads_started":1}
I20260619 08:27:14.397688 10620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:14.397401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:40771)
0619 08:27:14.397576 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:14.397627 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":112,"threads_started":1}
I20260619 08:27:15.310111 10621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:15.309862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:56001)
0619 08:27:15.309998 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:15.310048 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":59,"threads_started":1}
I20260619 08:27:15.310129 10622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:15.309883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:39525)
0619 08:27:15.310049 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:15.310087 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260619 08:27:15.398305 10623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:15.398099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:49905)
0619 08:27:15.398222 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:15.398259 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260619 08:27:16.310925 10624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:16.310625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:47675)
0619 08:27:16.310793 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:16.310852 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":75,"threads_started":1}
I20260619 08:27:16.310966 10625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:16.310724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:52641)
0619 08:27:16.310880 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:16.310919 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260619 08:27:16.311357 10624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:16.311238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:34963)
0619 08:27:16.311299 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:16.311333 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 08:27:16.399262 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:16.399003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:35701)
0619 08:27:16.399170 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:16.399214 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
I20260619 08:27:17.311827 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:17.311553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:48237)
0619 08:27:17.311706 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:17.311749 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":91,"threads_started":1}
I20260619 08:27:17.311995 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:17.311798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:45179)
0619 08:27:17.311889 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:17.311940 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":41,"threads_started":1}
I20260619 08:27:17.400051 10629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:17.399807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:45701)
0619 08:27:17.399965 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:17.400004 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260619 08:27:18.312595 10630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:18.312364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:56827)
0619 08:27:18.312512 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:18.312548 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260619 08:27:18.312606 10631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:18.312396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:41177)
0619 08:27:18.312525 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:18.312565 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260619 08:27:18.400722 10632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:18.400518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:57481)
0619 08:27:18.400640 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:18.400674 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260619 08:27:19.313366 10633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:19.313122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:32905)
0619 08:27:19.313284 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:19.313323 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":63,"threads_started":1}
I20260619 08:27:19.313365 10634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:19.313120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:32991)
0619 08:27:19.313277 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:19.313317 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260619 08:27:19.401590 10635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:19.401349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:35857)
0619 08:27:19.401495 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:19.401540 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":71,"threads_started":1}
I20260619 08:27:20.314221 10636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:20.313933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:36061)
0619 08:27:20.314098 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:20.314143 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":105,"threads_started":1}
I20260619 08:27:20.314200 10637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:20.313933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:58853)
0619 08:27:20.314073 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:20.314135 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260619 08:27:20.402626 10638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:20.402317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:37957)
0619 08:27:20.402512 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:20.402570 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":95,"threads_started":1}
I20260619 08:27:21.315114 10639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:21.314832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:40993)
0619 08:27:21.315001 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:21.315049 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
I20260619 08:27:21.315167 10640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:21.314895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:39507)
0619 08:27:21.315047 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:21.315112 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260619 08:27:21.403401 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:21.403145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:44235)
0619 08:27:21.403290 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:21.403354 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260619 08:27:22.315929 10642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:22.315666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:54629)
0619 08:27:22.315809 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:22.315860 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":60,"threads_started":1}
I20260619 08:27:22.316040 10643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:22.315798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:38701)
0619 08:27:22.315934 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:22.315970 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
I20260619 08:27:22.404268 10644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:22.403982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:34653)
0619 08:27:22.404132 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:22.404220 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":90,"threads_started":1}
I20260619 08:27:23.316723 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:23.316486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:42183)
0619 08:27:23.316634 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:23.316672 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":103,"threads_started":1}
I20260619 08:27:23.316723 10645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:23.316486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:42029)
0619 08:27:23.316631 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:23.316678 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":62,"threads_started":1}
I20260619 08:27:23.405020 10647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:23.404757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:42543)
0619 08:27:23.404907 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:23.404971 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
I20260619 08:27:24.317485 10648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:24.317224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:43701)
0619 08:27:24.317379 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:24.317428 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":66,"threads_started":1}
I20260619 08:27:24.317485 10649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:24.317220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:36693)
0619 08:27:24.317406 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:24.317435 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":102,"threads_started":1}
I20260619 08:27:24.405738 10650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:24.405504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:52383)
0619 08:27:24.405640 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:24.405682 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
I20260619 08:27:25.318346 10652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:25.318078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.129:35069)
0619 08:27:25.318259 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:25.318298 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":81,"threads_started":1}
I20260619 08:27:25.318346 10651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:25.318078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.131:55343)
0619 08:27:25.318252 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:25.318301 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":79,"threads_started":1}
I20260619 08:27:25.406435 10653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:25.406234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46391 (local address 127.25.127.130:40811)
0619 08:27:25.406349 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:25.406386 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.190:46391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
I20260619 08:27:25.955243 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 10151
I20260619 08:27:25.960417 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 10320
I20260619 08:27:25.965333 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 10456
2026-06-19T08:27:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58708 ms)
[----------] 30 tests from SecurityITest (233468 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:26 dist-test-slave-5rhg krb5kdc[10656](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:26 dist-test-slave-5rhg krb5kdc[10656](info): set up 2 sockets
Jun 19 08:27:26 dist-test-slave-5rhg krb5kdc[10656](info): commencing operation
krb5kdc: starting...
W20260619 08:27:27.040781 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:27 dist-test-slave-5rhg krb5kdc[10656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:27:27.158613 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:43993
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:43993
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:27.242156 10669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:27.242393 10669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:27.242417 10669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:27.243721 10669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:27.243765 10669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:27.243780 10669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:27.243831 10669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:27.243856 10669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:27.245375 10669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:43993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:43993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:27.245645 10669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:27.245895 10669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:27.251021 10677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:27.251013 10674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:27.251013 10675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:27.251040 10669 server_base.cc:1034] running on GCE node
I20260619 08:27:27.251546 10669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:27.251854 10669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:27.253001 10669 hybrid_clock.cc:648] HybridClock initialized: now 1781857647252981 us; error 42 us; skew 500 ppm
Jun 19 08:27:27 dist-test-slave-5rhg krb5kdc[10656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857647, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:27.255637 10669 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:27:27.256084 10669 webserver.cc:460] Webserver started at http://127.25.127.190:34969/ using document root <none> and password file <none>
I20260619 08:27:27.256335 10669 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:27.256404 10669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:27.256551 10669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:27.257679 10669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "a0b4e78c47154015a88b65927777269a"
format_stamp: "Formatted at 2026-06-19 08:27:27 on dist-test-slave-5rhg"
server_key: "cf5b6ea7c7811bc8344884a1902b431c"
server_key_iv: "8ae2efefb52eddd673b9ee4f08e99173"
server_key_version: "encryptionkey@0"
I20260619 08:27:27.258047 10669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "a0b4e78c47154015a88b65927777269a"
format_stamp: "Formatted at 2026-06-19 08:27:27 on dist-test-slave-5rhg"
server_key: "cf5b6ea7c7811bc8344884a1902b431c"
server_key_iv: "8ae2efefb52eddd673b9ee4f08e99173"
server_key_version: "encryptionkey@0"
I20260619 08:27:27.259426 10669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:27:27.260253 10684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:27.260432 10669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:27.260531 10669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "a0b4e78c47154015a88b65927777269a"
format_stamp: "Formatted at 2026-06-19 08:27:27 on dist-test-slave-5rhg"
server_key: "cf5b6ea7c7811bc8344884a1902b431c"
server_key_iv: "8ae2efefb52eddd673b9ee4f08e99173"
server_key_version: "encryptionkey@0"
I20260619 08:27:27.260617 10669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:27.271454 10669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:27.274010 10669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:27.274180 10669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:27.278337 10669 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:43993
I20260619 08:27:27.278385 10736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:43993 every 8 connection(s)
I20260619 08:27:27.278923 10669 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:27.279600 10737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:27.282039 10737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a: Bootstrap starting.
I20260619 08:27:27.282655 10737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:27.282960 10737 log.cc:824] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:27.283731 10737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a: No bootstrap required, opened a new log
I20260619 08:27:27.283839 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 10669
I20260619 08:27:27.283952 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:27.284173 26110 external_mini_cluster.cc:1428] Setting key e571448dedab31e21e62ae8bba016936
I20260619 08:27:27.285820 10737 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b4e78c47154015a88b65927777269a" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 43993 } }
I20260619 08:27:27.286005 10737 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:27.286089 10737 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0b4e78c47154015a88b65927777269a, State: Initialized, Role: FOLLOWER
I20260619 08:27:27.286269 10737 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b4e78c47154015a88b65927777269a" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 43993 } }
I20260619 08:27:27.286365 10737 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:27.286406 10737 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:27.286489 10737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:27.287173 10737 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b4e78c47154015a88b65927777269a" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 43993 } }
I20260619 08:27:27.287268 10737 leader_election.cc:302] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0b4e78c47154015a88b65927777269a; no voters:
I20260619 08:27:27.287490 10737 leader_election.cc:288] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:27.287513 10742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:27.287622 10742 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a [term 1 LEADER]: Becoming Leader. State: Replica: a0b4e78c47154015a88b65927777269a, State: Running, Role: LEADER
I20260619 08:27:27.287709 10742 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b4e78c47154015a88b65927777269a" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 43993 } }
I20260619 08:27:27.287781 10737 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:27.288059 10742 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0b4e78c47154015a88b65927777269a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b4e78c47154015a88b65927777269a" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 43993 } } }
I20260619 08:27:27.288245 10742 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:27.288453 10742 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0b4e78c47154015a88b65927777269a. Latest consensus state: current_term: 1 leader_uuid: "a0b4e78c47154015a88b65927777269a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b4e78c47154015a88b65927777269a" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 43993 } } }
I20260619 08:27:27.288579 10742 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:27.288944 10750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:27.289520 10750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:27.291056 10750 catalog_manager.cc:1347] Generated new cluster ID: 1a26c5c9e7bc4d868e4454bb8c9df15f
I20260619 08:27:27.291110 10750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:27:27 dist-test-slave-5rhg krb5kdc[10656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:27:27.296334 10741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.284719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50738 (local address 127.25.127.190:43993)
0619 08:27:27.284897 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:27.284906 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:27.284931 (+ 25us) server_negotiation.cc:407] Connection header received
0619 08:27:27.285807 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:27.285821 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:27.285875 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:27.285992 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:27.286784 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.288109 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.289341 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.290048 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.290159 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.292799 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:27.292803 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:27.292803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:27.292816 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:27.294579 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:27.294990 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:27.294994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:27.294994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:27.295111 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:27.295333 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:27.295334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:27.295334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:27.295588 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:27.295768 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:27.295868 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 08:27:27.296025 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260619 08:27:27.298756 10750 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:27.300501 10750 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:27.321944 10750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a: Generated new TSK 0
I20260619 08:27:27.322216 10750 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
I20260619 08:27:27.353237 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:43993
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:27.473136 10764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:27.473431 10764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:27.473469 10764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:27.475237 10764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:27.475298 10764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:27.475340 10764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:27:27.477547 10764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:43993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:27.477843 10764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:27.478168 10764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:27.483841 10772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:27.483911 10764 server_base.cc:1034] running on GCE node
W20260619 08:27:27.483841 10769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:27.484007 10770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:27.484395 10764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:27.484663 10764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:27.485819 10764 hybrid_clock.cc:648] HybridClock initialized: now 1781857647485799 us; error 41 us; skew 500 ppm
Jun 19 08:27:27 dist-test-slave-5rhg krb5kdc[10656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857647, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:27.489107 10764 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:27:27.489732 10764 webserver.cc:460] Webserver started at http://127.25.127.129:46549/ using document root <none> and password file <none>
I20260619 08:27:27.489984 10764 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:27.490031 10764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:27.490178 10764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:27.491276 10764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "e5001bda8ce04fab8bb0cea720d6446a"
format_stamp: "Formatted at 2026-06-19 08:27:27 on dist-test-slave-5rhg"
server_key: "acda072c54bb54f53da700c5302dd7b3"
server_key_iv: "de3ef27cd3b7cb6ca3e0f46380451532"
server_key_version: "encryptionkey@0"
I20260619 08:27:27.491654 10764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "e5001bda8ce04fab8bb0cea720d6446a"
format_stamp: "Formatted at 2026-06-19 08:27:27 on dist-test-slave-5rhg"
server_key: "acda072c54bb54f53da700c5302dd7b3"
server_key_iv: "de3ef27cd3b7cb6ca3e0f46380451532"
server_key_version: "encryptionkey@0"
I20260619 08:27:27.493098 10764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:27:27.493884 10779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:27.494120 10764 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:27.494231 10764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "e5001bda8ce04fab8bb0cea720d6446a"
format_stamp: "Formatted at 2026-06-19 08:27:27 on dist-test-slave-5rhg"
server_key: "acda072c54bb54f53da700c5302dd7b3"
server_key_iv: "de3ef27cd3b7cb6ca3e0f46380451532"
server_key_version: "encryptionkey@0"
I20260619 08:27:27.494304 10764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:27.514935 10764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:27.517483 10764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:27.517643 10764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:27.517908 10764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:27.518299 10764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:27.518359 10764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:27.518404 10764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:27.518432 10764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:27.524312 10764 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:43447
I20260619 08:27:27.524354 10893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:43447 every 8 connection(s)
I20260619 08:27:27.524689 10764 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:27:27.529366 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 10764
I20260619 08:27:27.529518 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:27:27.529728 26110 external_mini_cluster.cc:1428] Setting key 86f02d067e917edf178d2aef1a07fd99
Jun 19 08:27:27 dist-test-slave-5rhg krb5kdc[10656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857647, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:27:27.534880 10741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.525458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:50265 (local address 127.25.127.190:43993)
0619 08:27:27.525527 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:27.525529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:27.525681 (+ 152us) server_negotiation.cc:407] Connection header received
0619 08:27:27.526212 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:27.526214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:27.526230 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:27.526308 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:27.527304 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.527823 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.528861 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.529081 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.529140 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.531616 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:27.531618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:27.531618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:27.531630 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:27.533206 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:27.533710 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:27.533711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:27.533711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:27.533762 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:27.534091 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:27.534093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:27.534093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:27.534281 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:27.534366 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:27.534731 (+ 365us) server_negotiation.cc:299] Negotiation successful
0619 08:27:27.534812 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:27:27.535172 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.525377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:43993 (local address 127.25.127.129:50265)
0619 08:27:27.525547 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:27.525591 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:27.526092 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:27.526385 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:27.526388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:27.527001 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:27.527204 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:27.527209 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.527904 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.527904 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:27.528784 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:27.528786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.529157 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.529158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:27.529262 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.529908 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:27:27.529922 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:27.531522 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:27.533293 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:27.533300 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:27.533303 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:27.533633 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:27.533826 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:27.533827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:27.533828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:27.533984 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:27.534371 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:27.534379 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:27.534573 (+ 194us) client_negotiation.cc:769] Sending connection context
0619 08:27:27.534702 (+ 129us) client_negotiation.cc:240] Negotiation successful
0619 08:27:27.534894 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260619 08:27:27.535848 10894 heartbeater.cc:347] Connected to a master server at 127.25.127.190:43993
I20260619 08:27:27.535969 10894 heartbeater.cc:464] Registering TS with master...
I20260619 08:27:27.536216 10894 heartbeater.cc:511] Master 127.25.127.190:43993 requested a full tablet report, sending...
I20260619 08:27:27.536829 10700 ts_manager.cc:194] Registered new tserver with Master: e5001bda8ce04fab8bb0cea720d6446a (127.25.127.129:43447)
I20260619 08:27:27.537461 10700 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:50265
WARNING: no policy specified for kudu/127.25.127.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
I20260619 08:27:27.555601 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:43993
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:27.647581 10899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:27.647804 10899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:27.647830 10899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:27.649170 10899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:27.649217 10899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:27.649250 10899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:27:27.650877 10899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:43993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:27.651173 10899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:27.651434 10899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:27.657665 10907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:27.657696 10904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:27.657665 10905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:27.657717 10899 server_base.cc:1034] running on GCE node
I20260619 08:27:27.658080 10899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:27.658363 10899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:27.659545 10899 hybrid_clock.cc:648] HybridClock initialized: now 1781857647659527 us; error 29 us; skew 500 ppm
Jun 19 08:27:27 dist-test-slave-5rhg krb5kdc[10656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857647, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:27.662197 10899 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:27:27.662670 10899 webserver.cc:460] Webserver started at http://127.25.127.130:45761/ using document root <none> and password file <none>
I20260619 08:27:27.662943 10899 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:27.663010 10899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:27.663184 10899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:27.664391 10899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "d135c70227574b02b713a49a83fe5556"
format_stamp: "Formatted at 2026-06-19 08:27:27 on dist-test-slave-5rhg"
server_key: "94de494c743b874365b7337afc576601"
server_key_iv: "1f3c4aec54944d605bbaddb2709af8b0"
server_key_version: "encryptionkey@0"
I20260619 08:27:27.664764 10899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "d135c70227574b02b713a49a83fe5556"
format_stamp: "Formatted at 2026-06-19 08:27:27 on dist-test-slave-5rhg"
server_key: "94de494c743b874365b7337afc576601"
server_key_iv: "1f3c4aec54944d605bbaddb2709af8b0"
server_key_version: "encryptionkey@0"
I20260619 08:27:27.666131 10899 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:27:27.666862 10914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:27.667039 10899 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:27:27.667143 10899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "d135c70227574b02b713a49a83fe5556"
format_stamp: "Formatted at 2026-06-19 08:27:27 on dist-test-slave-5rhg"
server_key: "94de494c743b874365b7337afc576601"
server_key_iv: "1f3c4aec54944d605bbaddb2709af8b0"
server_key_version: "encryptionkey@0"
I20260619 08:27:27.667223 10899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:27.679927 10899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:27.682065 10899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:27.682238 10899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:27.682489 10899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:27.682821 10899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:27.682873 10899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:27.682914 10899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:27.682941 10899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:27.689227 10899 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:36943
I20260619 08:27:27.689272 11028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:36943 every 8 connection(s)
I20260619 08:27:27.689600 10899 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:27:27.691105 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 10899
I20260619 08:27:27.691236 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:27:27.691475 26110 external_mini_cluster.cc:1428] Setting key bef463665e11ad694f9d1950d67d4c2b
Jun 19 08:27:27 dist-test-slave-5rhg krb5kdc[10656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857647, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.127.131@KRBTEST.COM; defaulting to no policy
I20260619 08:27:27.700837 10741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.690340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:56869 (local address 127.25.127.190:43993)
0619 08:27:27.690406 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:27.690408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:27.690597 (+ 189us) server_negotiation.cc:407] Connection header received
0619 08:27:27.691316 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:27.691317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:27.691332 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:27.691408 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:27.692791 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.693302 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.694498 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.694789 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.694849 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.697695 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:27.697697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:27.697697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:27.697706 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:27.699375 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:27.699780 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:27.699782 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:27.699782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:27.699848 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:27.700108 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:27.700110 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:27.700110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:27.700295 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:27.700374 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:27.700626 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 08:27:27.700727 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 08:27:27.700989 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.690321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:43993 (local address 127.25.127.130:56869)
0619 08:27:27.690476 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:27.690519 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:27.691157 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:27.691492 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:27.691497 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:27.692342 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:27.692684 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:27.692692 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.693408 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.693410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:27.694401 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:27.694404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.694884 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.694886 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:27.695047 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.695608 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:27:27.695620 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:27.697592 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:27.699457 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:27.699465 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:27.699468 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:27.699697 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:27.699936 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:27.699936 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:27.699937 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:27.700039 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:27.700364 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:27.700369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:27.700491 (+ 122us) client_negotiation.cc:769] Sending connection context
0619 08:27:27.700578 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 08:27:27.700770 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260619 08:27:27.701475 11029 heartbeater.cc:347] Connected to a master server at 127.25.127.190:43993
I20260619 08:27:27.701563 11029 heartbeater.cc:464] Registering TS with master...
I20260619 08:27:27.701779 11029 heartbeater.cc:511] Master 127.25.127.190:43993 requested a full tablet report, sending...
I20260619 08:27:27.702209 10700 ts_manager.cc:194] Registered new tserver with Master: d135c70227574b02b713a49a83fe5556 (127.25.127.130:36943)
I20260619 08:27:27.702658 10700 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:56869
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
I20260619 08:27:27.718329 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:43993
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:27.802132 11034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:27.802380 11034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:27.802423 11034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:27.803638 11034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:27.803706 11034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:27.803754 11034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:27:27.805364 11034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:43993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:27.805660 11034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:27.805913 11034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:27.811065 11042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:27.811092 11034 server_base.cc:1034] running on GCE node
W20260619 08:27:27.811065 11039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:27.811095 11040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:27.811542 11034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:27.811815 11034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:27.813006 11034 hybrid_clock.cc:648] HybridClock initialized: now 1781857647812997 us; error 36 us; skew 500 ppm
Jun 19 08:27:27 dist-test-slave-5rhg krb5kdc[10656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857647, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:27.815635 11034 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:27:27.816234 11034 webserver.cc:460] Webserver started at http://127.25.127.131:39787/ using document root <none> and password file <none>
I20260619 08:27:27.816470 11034 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:27.816540 11034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:27.816687 11034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:27.817826 11034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "e5443850d7cc418d9d2355f19893f678"
format_stamp: "Formatted at 2026-06-19 08:27:27 on dist-test-slave-5rhg"
server_key: "80eabdd1d20e9fab0232a33f64199c8d"
server_key_iv: "1c914879774edd0590fbb84f4e90ff1d"
server_key_version: "encryptionkey@0"
I20260619 08:27:27.818220 11034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "e5443850d7cc418d9d2355f19893f678"
format_stamp: "Formatted at 2026-06-19 08:27:27 on dist-test-slave-5rhg"
server_key: "80eabdd1d20e9fab0232a33f64199c8d"
server_key_iv: "1c914879774edd0590fbb84f4e90ff1d"
server_key_version: "encryptionkey@0"
I20260619 08:27:27.819734 11034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:27:27.820631 11049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:27.820825 11034 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:27:27.820945 11034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "e5443850d7cc418d9d2355f19893f678"
format_stamp: "Formatted at 2026-06-19 08:27:27 on dist-test-slave-5rhg"
server_key: "80eabdd1d20e9fab0232a33f64199c8d"
server_key_iv: "1c914879774edd0590fbb84f4e90ff1d"
server_key_version: "encryptionkey@0"
I20260619 08:27:27.821029 11034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:27.828815 11034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:27.830766 11034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:27.830931 11034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:27.831169 11034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:27.831503 11034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:27.831552 11034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:27.831588 11034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:27.831609 11034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:27.837677 11034 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:40229
I20260619 08:27:27.837725 11163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:40229 every 8 connection(s)
I20260619 08:27:27.838045 11034 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:27:27.843349 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 11034
I20260619 08:27:27.843434 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:27:27.843640 26110 external_mini_cluster.cc:1428] Setting key aac097fbf824b581281889154e33b6a7
Jun 19 08:27:27 dist-test-slave-5rhg krb5kdc[10656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857647, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:27:27.847666 10741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.838767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:41723 (local address 127.25.127.190:43993)
0619 08:27:27.838830 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:27.838832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:27.839003 (+ 171us) server_negotiation.cc:407] Connection header received
0619 08:27:27.839500 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:27.839501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:27.839510 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:27.839586 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:27.840689 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.841205 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.842238 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.842430 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.842495 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.845072 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:27.845074 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:27.845074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:27.845080 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:27.846401 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:27.846812 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:27.846813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:27.846813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:27.846868 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:27.847086 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:27.847087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:27.847087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:27.847203 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:27.847256 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:27.847508 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 08:27:27.847592 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:27:27.847921 11166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.838716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:43993 (local address 127.25.127.131:41723)
0619 08:27:27.838875 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:27.838917 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:27.839414 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:27.839640 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:27.839643 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:27.840336 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:27.840545 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:27.840551 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.841302 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.841304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:27.842136 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:27.842139 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.842513 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.842514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:27.842622 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.843204 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:27:27.843213 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:27.844987 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:27.846487 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:27.846495 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:27.846496 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:27.846727 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:27.846929 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:27.846929 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:27.846929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:27.847029 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:27.847266 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:27.847271 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:27.847390 (+ 119us) client_negotiation.cc:769] Sending connection context
0619 08:27:27.847477 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 08:27:27.847647 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260619 08:27:27.848490 11164 heartbeater.cc:347] Connected to a master server at 127.25.127.190:43993
I20260619 08:27:27.848600 11164 heartbeater.cc:464] Registering TS with master...
I20260619 08:27:27.848783 11164 heartbeater.cc:511] Master 127.25.127.190:43993 requested a full tablet report, sending...
I20260619 08:27:27.849156 10700 ts_manager.cc:194] Registered new tserver with Master: e5443850d7cc418d9d2355f19893f678 (127.25.127.131:40229)
I20260619 08:27:27.849511 10700 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:41723
I20260619 08:27:27.855304 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:27:27.862916 10741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.856828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48728 (local address 127.25.127.190:43993)
0619 08:27:27.856909 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:27.856911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:27.856934 (+ 23us) server_negotiation.cc:407] Connection header received
0619 08:27:27.856992 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:27.856993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:27.857001 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:27.857040 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:27.857570 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.858041 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.858778 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.858955 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.859002 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.860289 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:27.860290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:27.860291 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:27.860298 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:27.861805 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:27.862142 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:27.862142 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:27.862143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:27.862198 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:27.862368 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:27.862368 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:27.862369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:27.862528 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:27.862605 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:27.862711 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 08:27:27.862824 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 08:27:27.864671 10700 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:27:27.866268 10700 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:27:27.873205 11175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.868828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:40229 (local address 127.0.0.1:41422)
0619 08:27:27.869132 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:27.869142 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:27.869227 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:27.869478 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:27.869482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:27.869485 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:27.869579 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:27.869581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.870672 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.870673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:27.871916 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:27.871919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.872873 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.872874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:27.872956 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.872972 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 08:27:27.873040 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 08:27:27.873101 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":97,"threads_started":1}
I20260619 08:27:27.873333 11177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.868945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36046 (local address 127.25.127.129:43447)
0619 08:27:27.869476 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:27.869480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:27.869490 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:27:27.869609 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:27.869611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:27.869627 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:27.869852 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:27.869930 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.871083 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.872400 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.872959 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.873040 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.873096 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:27.873189 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 08:27:27.873242 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":85,"threads_started":1}
I20260619 08:27:27.873685 11178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.868946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41422 (local address 127.25.127.131:40229)
0619 08:27:27.869303 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:27.869309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:27.869319 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:27:27.869363 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:27.869365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:27.869377 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:27.869464 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:27.869690 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.870569 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.872007 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.872782 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.872860 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.872914 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:27.873586 (+ 672us) server_negotiation.cc:299] Negotiation successful
0619 08:27:27.873621 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":102,"threads_started":1}
I20260619 08:27:27.873744 11180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.869264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44244 (local address 127.25.127.130:36943)
0619 08:27:27.869411 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:27.869416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:27.869548 (+ 132us) server_negotiation.cc:407] Connection header received
0619 08:27:27.869594 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:27.869597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:27.869613 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:27.869852 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:27.869931 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.871092 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.872446 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.873191 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.873554 (+ 363us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.873598 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:27.873621 (+ 23us) server_negotiation.cc:299] Negotiation successful
0619 08:27:27.873654 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
I20260619 08:27:27.873317 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.868822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:43447 (local address 127.0.0.1:36046)
0619 08:27:27.869010 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:27.869056 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:27.869125 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:27.869738 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:27.869739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:27.869741 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:27.869775 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:27.869777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.871222 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.871223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:27.872316 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:27.872319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.873047 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.873048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:27.873113 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.873117 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:27:27.873177 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 08:27:27.873236 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":22,"thread_start_us":69,"threads_started":1}
I20260619 08:27:27.874554 11097 tablet_service.cc:1467] Processing CreateTablet for tablet a7480d02ac764a7e863d6b78c7490033 (DEFAULT_TABLE table=test-table [id=e2dfac215fcf4e3d9a59ad5bcf6b4dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:27:27.874555 10827 tablet_service.cc:1467] Processing CreateTablet for tablet a7480d02ac764a7e863d6b78c7490033 (DEFAULT_TABLE table=test-table [id=e2dfac215fcf4e3d9a59ad5bcf6b4dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:27:27.874930 10827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7480d02ac764a7e863d6b78c7490033. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:27.874931 11097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7480d02ac764a7e863d6b78c7490033. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:27.873488 11179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.869039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:36943 (local address 127.0.0.1:44244)
0619 08:27:27.869149 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:27.869161 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:27.869222 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:27.869719 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:27.869720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:27.869722 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:27.869775 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:27.869780 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.871231 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.871232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:27.872350 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:27.872353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.873280 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.873281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:27.873346 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.873351 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:27:27.873380 (+ 29us) client_negotiation.cc:240] Negotiation successful
0619 08:27:27.873422 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":62,"thread_start_us":65,"threads_started":1}
I20260619 08:27:27.877022 10962 tablet_service.cc:1467] Processing CreateTablet for tablet a7480d02ac764a7e863d6b78c7490033 (DEFAULT_TABLE table=test-table [id=e2dfac215fcf4e3d9a59ad5bcf6b4dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:27:27.877373 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7480d02ac764a7e863d6b78c7490033. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:27.878392 11181 tablet_bootstrap.cc:492] T a7480d02ac764a7e863d6b78c7490033 P e5001bda8ce04fab8bb0cea720d6446a: Bootstrap starting.
I20260619 08:27:27.878628 11182 tablet_bootstrap.cc:492] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678: Bootstrap starting.
I20260619 08:27:27.879362 11182 tablet_bootstrap.cc:654] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:27.879945 11182 log.cc:824] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:27.880722 11184 tablet_bootstrap.cc:492] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556: Bootstrap starting.
I20260619 08:27:27.880844 11182 tablet_bootstrap.cc:492] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678: No bootstrap required, opened a new log
I20260619 08:27:27.880923 11182 ts_tablet_manager.cc:1397] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 08:27:27.881055 11181 tablet_bootstrap.cc:654] T a7480d02ac764a7e863d6b78c7490033 P e5001bda8ce04fab8bb0cea720d6446a: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:27.881433 11184 tablet_bootstrap.cc:654] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:27.881631 11181 log.cc:824] T a7480d02ac764a7e863d6b78c7490033 P e5001bda8ce04fab8bb0cea720d6446a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:27.881984 11184 log.cc:824] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:27.882433 11181 tablet_bootstrap.cc:492] T a7480d02ac764a7e863d6b78c7490033 P e5001bda8ce04fab8bb0cea720d6446a: No bootstrap required, opened a new log
I20260619 08:27:27.882503 11181 ts_tablet_manager.cc:1397] T a7480d02ac764a7e863d6b78c7490033 P e5001bda8ce04fab8bb0cea720d6446a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 08:27:27.882737 11184 tablet_bootstrap.cc:492] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556: No bootstrap required, opened a new log
I20260619 08:27:27.882812 11184 ts_tablet_manager.cc:1397] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:27:27.883081 11182 raft_consensus.cc:348] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5001bda8ce04fab8bb0cea720d6446a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43447 } } peers { permanent_uuid: "e5443850d7cc418d9d2355f19893f678" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 40229 } } peers { permanent_uuid: "d135c70227574b02b713a49a83fe5556" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36943 } }
I20260619 08:27:27.883260 11182 raft_consensus.cc:374] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:27.883299 11182 raft_consensus.cc:729] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5443850d7cc418d9d2355f19893f678, State: Initialized, Role: FOLLOWER
I20260619 08:27:27.883435 11182 consensus_queue.cc:260] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5001bda8ce04fab8bb0cea720d6446a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43447 } } peers { permanent_uuid: "e5443850d7cc418d9d2355f19893f678" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 40229 } } peers { permanent_uuid: "d135c70227574b02b713a49a83fe5556" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36943 } }
I20260619 08:27:27.883714 11182 ts_tablet_manager.cc:1428] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 08:27:27.883865 11164 heartbeater.cc:503] Master 127.25.127.190:43993 was elected leader, sending a full tablet report...
I20260619 08:27:27.884456 11181 raft_consensus.cc:348] T a7480d02ac764a7e863d6b78c7490033 P e5001bda8ce04fab8bb0cea720d6446a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5001bda8ce04fab8bb0cea720d6446a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43447 } } peers { permanent_uuid: "e5443850d7cc418d9d2355f19893f678" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 40229 } } peers { permanent_uuid: "d135c70227574b02b713a49a83fe5556" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36943 } }
I20260619 08:27:27.884639 11181 raft_consensus.cc:374] T a7480d02ac764a7e863d6b78c7490033 P e5001bda8ce04fab8bb0cea720d6446a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:27.884701 11181 raft_consensus.cc:729] T a7480d02ac764a7e863d6b78c7490033 P e5001bda8ce04fab8bb0cea720d6446a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5001bda8ce04fab8bb0cea720d6446a, State: Initialized, Role: FOLLOWER
I20260619 08:27:27.884837 11181 consensus_queue.cc:260] T a7480d02ac764a7e863d6b78c7490033 P e5001bda8ce04fab8bb0cea720d6446a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5001bda8ce04fab8bb0cea720d6446a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43447 } } peers { permanent_uuid: "e5443850d7cc418d9d2355f19893f678" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 40229 } } peers { permanent_uuid: "d135c70227574b02b713a49a83fe5556" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36943 } }
I20260619 08:27:27.885118 11181 ts_tablet_manager.cc:1428] T a7480d02ac764a7e863d6b78c7490033 P e5001bda8ce04fab8bb0cea720d6446a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:27:27.885010 11184 raft_consensus.cc:348] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5001bda8ce04fab8bb0cea720d6446a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43447 } } peers { permanent_uuid: "e5443850d7cc418d9d2355f19893f678" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 40229 } } peers { permanent_uuid: "d135c70227574b02b713a49a83fe5556" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36943 } }
I20260619 08:27:27.885170 11184 raft_consensus.cc:374] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:27.885234 11184 raft_consensus.cc:729] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d135c70227574b02b713a49a83fe5556, State: Initialized, Role: FOLLOWER
I20260619 08:27:27.885246 10894 heartbeater.cc:503] Master 127.25.127.190:43993 was elected leader, sending a full tablet report...
I20260619 08:27:27.885368 11184 consensus_queue.cc:260] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5001bda8ce04fab8bb0cea720d6446a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43447 } } peers { permanent_uuid: "e5443850d7cc418d9d2355f19893f678" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 40229 } } peers { permanent_uuid: "d135c70227574b02b713a49a83fe5556" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36943 } }
I20260619 08:27:27.885638 11184 ts_tablet_manager.cc:1428] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:27:27.885653 11029 heartbeater.cc:503] Master 127.25.127.190:43993 was elected leader, sending a full tablet report...
W20260619 08:27:27.940286 11030 tablet.cc:2369] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:27:27.964761 11189 raft_consensus.cc:482] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:27:27.964891 11189 raft_consensus.cc:504] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5001bda8ce04fab8bb0cea720d6446a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43447 } } peers { permanent_uuid: "e5443850d7cc418d9d2355f19893f678" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 40229 } } peers { permanent_uuid: "d135c70227574b02b713a49a83fe5556" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36943 } }
I20260619 08:27:27.965236 11189 leader_election.cc:288] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5001bda8ce04fab8bb0cea720d6446a (127.25.127.129:43447), e5443850d7cc418d9d2355f19893f678 (127.25.127.131:40229)
I20260619 08:27:27.968451 11177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.965601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:34993 (local address 127.25.127.129:43447)
0619 08:27:27.965651 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:27.965654 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:27.965848 (+ 194us) server_negotiation.cc:407] Connection header received
0619 08:27:27.965884 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:27.965885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:27.965897 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:27.965972 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:27.966207 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.966715 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.967647 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.968045 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.968136 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.968147 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:27.968312 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 08:27:27.968364 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:27:27.968475 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.965492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:40229 (local address 127.25.127.130:50455)
0619 08:27:27.965574 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:27.965587 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:27.965638 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:27.965871 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:27.965872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:27.965874 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:27.965959 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:27.965962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.966683 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.966684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:27.967541 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:27.967543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.968185 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.968186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:27.968259 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.968262 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:27:27.968341 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 08:27:27.968379 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260619 08:27:27.968477 11178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.965615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:50455 (local address 127.25.127.131:40229)
0619 08:27:27.965671 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:27.965673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:27.965681 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:27:27.965713 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:27.965714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:27.965724 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:27.965806 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:27.966041 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.966589 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.967681 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.968054 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.968169 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.968177 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:27.968359 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 08:27:27.968403 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 08:27:27.968487 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.965492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:43447 (local address 127.25.127.130:34993)
0619 08:27:27.965745 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:27.965756 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:27.965815 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:27.966018 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:27.966019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:27.966021 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:27.966101 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:27.966103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.966815 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.966815 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:27.967550 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:27.967551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.968134 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.968135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:27.968232 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.968241 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 08:27:27.968284 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 08:27:27.968343 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":30,"thread_start_us":129,"threads_started":1}
I20260619 08:27:27.968870 11118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7480d02ac764a7e863d6b78c7490033" candidate_uuid: "d135c70227574b02b713a49a83fe5556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5443850d7cc418d9d2355f19893f678" is_pre_election: true
I20260619 08:27:27.969015 11118 raft_consensus.cc:2454] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d135c70227574b02b713a49a83fe5556 in term 0.
I20260619 08:27:27.969228 10915 leader_election.cc:302] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d135c70227574b02b713a49a83fe5556, e5443850d7cc418d9d2355f19893f678; no voters:
I20260619 08:27:27.969264 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7480d02ac764a7e863d6b78c7490033" candidate_uuid: "d135c70227574b02b713a49a83fe5556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5001bda8ce04fab8bb0cea720d6446a" is_pre_election: true
I20260619 08:27:27.969362 11189 raft_consensus.cc:2781] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:27:27.969448 11189 raft_consensus.cc:482] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:27:27.969448 10848 raft_consensus.cc:2454] T a7480d02ac764a7e863d6b78c7490033 P e5001bda8ce04fab8bb0cea720d6446a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d135c70227574b02b713a49a83fe5556 in term 0.
I20260619 08:27:27.969482 11189 raft_consensus.cc:3037] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:27.970271 11189 raft_consensus.cc:504] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5001bda8ce04fab8bb0cea720d6446a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43447 } } peers { permanent_uuid: "e5443850d7cc418d9d2355f19893f678" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 40229 } } peers { permanent_uuid: "d135c70227574b02b713a49a83fe5556" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36943 } }
I20260619 08:27:27.970420 11189 leader_election.cc:288] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [CANDIDATE]: Term 1 election: Requested vote from peers e5001bda8ce04fab8bb0cea720d6446a (127.25.127.129:43447), e5443850d7cc418d9d2355f19893f678 (127.25.127.131:40229)
I20260619 08:27:27.970639 11118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7480d02ac764a7e863d6b78c7490033" candidate_uuid: "d135c70227574b02b713a49a83fe5556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5443850d7cc418d9d2355f19893f678"
I20260619 08:27:27.970669 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7480d02ac764a7e863d6b78c7490033" candidate_uuid: "d135c70227574b02b713a49a83fe5556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5001bda8ce04fab8bb0cea720d6446a"
I20260619 08:27:27.970731 11118 raft_consensus.cc:3037] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:27.970746 10848 raft_consensus.cc:3037] T a7480d02ac764a7e863d6b78c7490033 P e5001bda8ce04fab8bb0cea720d6446a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:27.971629 11118 raft_consensus.cc:2454] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d135c70227574b02b713a49a83fe5556 in term 1.
I20260619 08:27:27.971629 10848 raft_consensus.cc:2454] T a7480d02ac764a7e863d6b78c7490033 P e5001bda8ce04fab8bb0cea720d6446a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d135c70227574b02b713a49a83fe5556 in term 1.
I20260619 08:27:27.971853 10916 leader_election.cc:302] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d135c70227574b02b713a49a83fe5556, e5001bda8ce04fab8bb0cea720d6446a; no voters:
I20260619 08:27:27.971985 11189 raft_consensus.cc:2781] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:27.972196 11189 raft_consensus.cc:686] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [term 1 LEADER]: Becoming Leader. State: Replica: d135c70227574b02b713a49a83fe5556, State: Running, Role: LEADER
I20260619 08:27:27.972301 11189 consensus_queue.cc:237] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5001bda8ce04fab8bb0cea720d6446a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43447 } } peers { permanent_uuid: "e5443850d7cc418d9d2355f19893f678" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 40229 } } peers { permanent_uuid: "d135c70227574b02b713a49a83fe5556" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36943 } }
I20260619 08:27:27.973112 10700 catalog_manager.cc:5495] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 reported cstate change: term changed from 0 to 1, leader changed from <none> to d135c70227574b02b713a49a83fe5556 (127.25.127.130). New cstate: current_term: 1 leader_uuid: "d135c70227574b02b713a49a83fe5556" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5001bda8ce04fab8bb0cea720d6446a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5443850d7cc418d9d2355f19893f678" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 40229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d135c70227574b02b713a49a83fe5556" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 36943 } health_report { overall_health: HEALTHY } } }
I20260619 08:27:27.988603 11180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:27.986125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44258 (local address 127.25.127.130:36943)
0619 08:27:27.986193 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:27.986195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:27.986203 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:27:27.986243 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:27.986243 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:27.986261 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:27.986322 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:27:27.986440 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.986936 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.987661 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:27.987946 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:27.987999 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:27.988212 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:27:27.988323 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:27:27.988389 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:27.988480 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 08:27:27.988506 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:27:27.990664 11118 raft_consensus.cc:1261] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678 [term 1 FOLLOWER]: Refusing update from remote peer d135c70227574b02b713a49a83fe5556: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:27:27.991020 10848 raft_consensus.cc:1261] T a7480d02ac764a7e863d6b78c7490033 P e5001bda8ce04fab8bb0cea720d6446a [term 1 FOLLOWER]: Refusing update from remote peer d135c70227574b02b713a49a83fe5556: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:27:27.991034 11191 consensus_queue.cc:1035] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5443850d7cc418d9d2355f19893f678" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 40229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:27:27.991302 11191 consensus_queue.cc:1035] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5001bda8ce04fab8bb0cea720d6446a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:27:27.993505 11194 mvcc.cc:205] Tried to move back new op lower bound from 7298488926166683648 to 7298488926095175680. Current Snapshot: MvccSnapshot[applied={T|T < 7298488926166683648}]
I20260619 08:27:27.994467 11196 mvcc.cc:205] Tried to move back new op lower bound from 7298488926166683648 to 7298488926095175680. Current Snapshot: MvccSnapshot[applied={T|T < 7298488926166683648}]
I20260619 08:27:27.996479 10701 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:27:27.996562 10701 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:27:27.997396 10701 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a0b4e78c47154015a88b65927777269a: Sending DeleteTablet for 3 replicas of tablet a7480d02ac764a7e863d6b78c7490033
I20260619 08:27:27.997833 10827 tablet_service.cc:1514] Processing DeleteTablet for tablet a7480d02ac764a7e863d6b78c7490033 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:27:27 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:36046
I20260619 08:27:27.997838 11097 tablet_service.cc:1514] Processing DeleteTablet for tablet a7480d02ac764a7e863d6b78c7490033 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:27:27 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:41422
I20260619 08:27:27.997869 10962 tablet_service.cc:1514] Processing DeleteTablet for tablet a7480d02ac764a7e863d6b78c7490033 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:27:27 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:44244
I20260619 08:27:27.998055 11203 tablet_replica.cc:330] T a7480d02ac764a7e863d6b78c7490033 P e5001bda8ce04fab8bb0cea720d6446a: stopping tablet replica
I20260619 08:27:27.998152 11203 raft_consensus.cc:2229] T a7480d02ac764a7e863d6b78c7490033 P e5001bda8ce04fab8bb0cea720d6446a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:27:27.998270 11203 raft_consensus.cc:2258] T a7480d02ac764a7e863d6b78c7490033 P e5001bda8ce04fab8bb0cea720d6446a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:27:27.998281 11205 tablet_replica.cc:330] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556: stopping tablet replica
I20260619 08:27:27.998436 11204 tablet_replica.cc:330] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678: stopping tablet replica
I20260619 08:27:27.998448 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 10764
I20260619 08:27:27.998531 11204 raft_consensus.cc:2229] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:27:27.998459 11205 raft_consensus.cc:2229] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:27:27.998628 11204 raft_consensus.cc:2258] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:27:27.998826 11205 raft_consensus.cc:2258] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:27:27.999243 11204 ts_tablet_manager.cc:1905] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:27:27.999904 11205 ts_tablet_manager.cc:1905] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:27:28.001027 11204 ts_tablet_manager.cc:1918] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:27:28.001109 11204 log.cc:1196] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/a7480d02ac764a7e863d6b78c7490033
I20260619 08:27:28.001391 11204 ts_tablet_manager.cc:1939] T a7480d02ac764a7e863d6b78c7490033 P e5443850d7cc418d9d2355f19893f678: Deleting consensus metadata
I20260619 08:27:28.001924 11205 ts_tablet_manager.cc:1918] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:27:28.001926 10685 catalog_manager.cc:4841] TS e5443850d7cc418d9d2355f19893f678 (127.25.127.131:40229): tablet a7480d02ac764a7e863d6b78c7490033 (table test-table [id=e2dfac215fcf4e3d9a59ad5bcf6b4dbb]) successfully deleted
I20260619 08:27:28.002070 11205 log.cc:1196] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/a7480d02ac764a7e863d6b78c7490033
I20260619 08:27:28.002357 11205 ts_tablet_manager.cc:1939] T a7480d02ac764a7e863d6b78c7490033 P d135c70227574b02b713a49a83fe5556: Deleting consensus metadata
I20260619 08:27:28.002720 10685 catalog_manager.cc:4841] TS d135c70227574b02b713a49a83fe5556 (127.25.127.130:36943): tablet a7480d02ac764a7e863d6b78c7490033 (table test-table [id=e2dfac215fcf4e3d9a59ad5bcf6b4dbb]) successfully deleted
W20260619 08:27:28.005333 10686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.127.129:43447 (error 108)
I20260619 08:27:28.005899 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.005744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:43447 (local address 127.0.0.1:36056)
0619 08:27:28.005802 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:28.005843 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.129:43447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 08:27:28.005966 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 10899
W20260619 08:27:28.007421 10686 catalog_manager.cc:4584] TS e5001bda8ce04fab8bb0cea720d6446a (127.25.127.129:43447): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a7480d02ac764a7e863d6b78c7490033: Network error: Client connection negotiation failed: client connection to 127.25.127.129:43447: connect: Connection refused (error 111)
I20260619 08:27:28.010991 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 11034
I20260619 08:27:28.015519 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 10669
2026-06-19T08:27:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2049 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T08:27:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:28Z Disabled control of system clock
I20260619 08:27:28.043318 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:40103
--webserver_interface=127.25.127.190
--webserver_port=0
--builtin_ntp_servers=127.25.127.148:42455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:40103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:27:28.128672 11210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:28.128896 11210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:28.128917 11210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:28.130023 11210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:28.130057 11210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:28.130070 11210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:28.130084 11210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:28.130095 11210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:28.131369 11210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:40103
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:40103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:28.131561 11210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:28.131798 11210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:28.136767 11216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:28.136761 11215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:28.136766 11218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:28.136943 11210 server_base.cc:1034] running on GCE node
I20260619 08:27:28.137234 11210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:28.137531 11210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:28.138680 11210 hybrid_clock.cc:648] HybridClock initialized: now 1781857648138670 us; error 34 us; skew 500 ppm
I20260619 08:27:28.139958 11210 webserver.cc:460] Webserver started at http://127.25.127.190:38597/ using document root <none> and password file <none>
I20260619 08:27:28.140282 11210 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:28.140334 11210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:28.140487 11210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:28.141587 11210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "899551f0617e4a33bf7d536794815643"
format_stamp: "Formatted at 2026-06-19 08:27:28 on dist-test-slave-5rhg"
server_key: "77476f9513ecb3204d8c19625e9558a4"
server_key_iv: "e0fff64f438b47b33a8e8d01ffe942dd"
server_key_version: "encryptionkey@0"
I20260619 08:27:28.141953 11210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "899551f0617e4a33bf7d536794815643"
format_stamp: "Formatted at 2026-06-19 08:27:28 on dist-test-slave-5rhg"
server_key: "77476f9513ecb3204d8c19625e9558a4"
server_key_iv: "e0fff64f438b47b33a8e8d01ffe942dd"
server_key_version: "encryptionkey@0"
I20260619 08:27:28.143280 11210 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:27:28.143996 11224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:28.144224 11210 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:27:28.144325 11210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "899551f0617e4a33bf7d536794815643"
format_stamp: "Formatted at 2026-06-19 08:27:28 on dist-test-slave-5rhg"
server_key: "77476f9513ecb3204d8c19625e9558a4"
server_key_iv: "e0fff64f438b47b33a8e8d01ffe942dd"
server_key_version: "encryptionkey@0"
I20260619 08:27:28.144410 11210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:28.152427 11210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:28.152719 11210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:28.152869 11210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:28.156776 11210 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:40103
I20260619 08:27:28.156809 11276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:40103 every 8 connection(s)
I20260619 08:27:28.157166 11210 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:28.157528 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 11210
I20260619 08:27:28.157635 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:28.157878 26110 external_mini_cluster.cc:1428] Setting key 5d6d45bf39c6990a67a6334874bf728e
I20260619 08:27:28.158386 11277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:28.161667 11277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643: Bootstrap starting.
I20260619 08:27:28.162361 11277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:28.162664 11277 log.cc:824] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:28.163534 11277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643: No bootstrap required, opened a new log
I20260619 08:27:28.163794 11280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.158487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32830 (local address 127.25.127.190:40103)
0619 08:27:28.158710 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:28.158717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:28.158743 (+ 26us) server_negotiation.cc:407] Connection header received
0619 08:27:28.159174 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:28.159181 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:28.159241 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:28.159358 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:28.159615 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.160584 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.161689 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.162151 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.162244 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.163056 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:28.163060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:28.163061 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:28.163273 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:28.163308 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:28.163320 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:28.163372 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:28.163475 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 08:27:28.163570 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260619 08:27:28.165572 11277 raft_consensus.cc:348] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "899551f0617e4a33bf7d536794815643" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40103 } }
I20260619 08:27:28.165781 11277 raft_consensus.cc:374] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:28.165834 11277 raft_consensus.cc:729] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 899551f0617e4a33bf7d536794815643, State: Initialized, Role: FOLLOWER
I20260619 08:27:28.165961 11277 consensus_queue.cc:260] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "899551f0617e4a33bf7d536794815643" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40103 } }
I20260619 08:27:28.166040 11277 raft_consensus.cc:388] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:28.166082 11277 raft_consensus.cc:482] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:28.166131 11277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:28.166684 11277 raft_consensus.cc:504] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "899551f0617e4a33bf7d536794815643" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40103 } }
I20260619 08:27:28.166818 11277 leader_election.cc:302] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 899551f0617e4a33bf7d536794815643; no voters:
I20260619 08:27:28.167025 11277 leader_election.cc:288] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:28.167083 11282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:28.167224 11282 raft_consensus.cc:686] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643 [term 1 LEADER]: Becoming Leader. State: Replica: 899551f0617e4a33bf7d536794815643, State: Running, Role: LEADER
I20260619 08:27:28.167311 11282 consensus_queue.cc:237] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "899551f0617e4a33bf7d536794815643" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40103 } }
I20260619 08:27:28.167351 11277 sys_catalog.cc:564] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:28.167589 11283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "899551f0617e4a33bf7d536794815643" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "899551f0617e4a33bf7d536794815643" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40103 } } }
I20260619 08:27:28.167625 11284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 899551f0617e4a33bf7d536794815643. Latest consensus state: current_term: 1 leader_uuid: "899551f0617e4a33bf7d536794815643" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "899551f0617e4a33bf7d536794815643" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40103 } } }
I20260619 08:27:28.167681 11283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:28.167702 11284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:28.167896 11287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:28.168493 11287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:28.170393 11287 catalog_manager.cc:1347] Generated new cluster ID: 269bb31910d443b789260f0180ddd0cd
I20260619 08:27:28.170455 11287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:28.199873 11287 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:28.200732 11287 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:28.207218 11287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643: Generated new TSK 0
I20260619 08:27:28.207432 11287 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:28.213336 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:40103
--builtin_ntp_servers=127.25.127.148:42455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:27:28.300282 11301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:28.300499 11301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:28.300522 11301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:28.301752 11301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:28.301797 11301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:28.301826 11301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:27:28.303201 11301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:40103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:28.303475 11301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:28.303802 11301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:28.308931 11309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:28.308964 11307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:28.308931 11306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:28.309180 11301 server_base.cc:1034] running on GCE node
I20260619 08:27:28.309412 11301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:28.309664 11301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:28.310848 11301 hybrid_clock.cc:648] HybridClock initialized: now 1781857648310842 us; error 37 us; skew 500 ppm
I20260619 08:27:28.312320 11301 webserver.cc:460] Webserver started at http://127.25.127.129:36325/ using document root <none> and password file <none>
I20260619 08:27:28.312644 11301 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:28.312723 11301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:28.312888 11301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:28.314136 11301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "8e7a590329ce4c4aa7368d9d438c2464"
format_stamp: "Formatted at 2026-06-19 08:27:28 on dist-test-slave-5rhg"
server_key: "4dee5f301083d3fd3e4a21a1aa9f0c58"
server_key_iv: "c4884358fa82de19afb0565ba723ab2f"
server_key_version: "encryptionkey@0"
I20260619 08:27:28.314509 11301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "8e7a590329ce4c4aa7368d9d438c2464"
format_stamp: "Formatted at 2026-06-19 08:27:28 on dist-test-slave-5rhg"
server_key: "4dee5f301083d3fd3e4a21a1aa9f0c58"
server_key_iv: "c4884358fa82de19afb0565ba723ab2f"
server_key_version: "encryptionkey@0"
I20260619 08:27:28.315838 11301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:27:28.316697 11315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:28.316879 11301 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:28.316977 11301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "8e7a590329ce4c4aa7368d9d438c2464"
format_stamp: "Formatted at 2026-06-19 08:27:28 on dist-test-slave-5rhg"
server_key: "4dee5f301083d3fd3e4a21a1aa9f0c58"
server_key_iv: "c4884358fa82de19afb0565ba723ab2f"
server_key_version: "encryptionkey@0"
I20260619 08:27:28.317090 11301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:28.347134 11301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:28.347517 11301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:28.347672 11301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:28.347920 11301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:28.348343 11301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:28.348381 11301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:28.348404 11301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:28.348450 11301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:28.354451 11301 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:43057
I20260619 08:27:28.354496 11429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:43057 every 8 connection(s)
I20260619 08:27:28.354832 11301 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:27:28.359064 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 11301
I20260619 08:27:28.359205 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:27:28.359403 26110 external_mini_cluster.cc:1428] Setting key 67c4751a3aa9f9d714600b8b80b52672
I20260619 08:27:28.360386 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:40103
--builtin_ntp_servers=127.25.127.148:42455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 08:27:28.360491 11280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.355607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:37727 (local address 127.25.127.190:40103)
0619 08:27:28.355677 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:28.355680 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:28.355985 (+ 305us) server_negotiation.cc:407] Connection header received
0619 08:27:28.356506 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:28.356507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:28.356522 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:28.356579 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:28.357282 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.357785 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.358796 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.359109 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.359189 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.360187 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:28.360189 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:28.360189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:28.360194 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:28.360210 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:28.360214 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:28.360289 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:28.360367 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 08:27:28.360410 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:27:28.360729 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.355607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:40103 (local address 127.25.127.129:37727)
0619 08:27:28.355847 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:28.355886 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:28.356420 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:28.356639 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:28.356642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:28.356650 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:28.357209 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:28.357214 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.357885 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.357886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:28.358717 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:28.358720 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.359223 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.359224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:28.359385 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.360006 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:27:28.360007 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:28.360037 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:27:28.360060 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:28.360290 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:28.360297 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:28.360300 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:27:28.360372 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 08:27:28.360482 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":91,"threads_started":1}
I20260619 08:27:28.361270 11430 heartbeater.cc:347] Connected to a master server at 127.25.127.190:40103
I20260619 08:27:28.361356 11430 heartbeater.cc:464] Registering TS with master...
I20260619 08:27:28.361565 11430 heartbeater.cc:511] Master 127.25.127.190:40103 requested a full tablet report, sending...
I20260619 08:27:28.362079 11241 ts_manager.cc:194] Registered new tserver with Master: 8e7a590329ce4c4aa7368d9d438c2464 (127.25.127.129:43057)
I20260619 08:27:28.362747 11241 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.127.129:37727
W20260619 08:27:28.442883 11433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:28.443084 11433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:28.443106 11433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:28.444254 11433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:28.444293 11433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:28.444319 11433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:27:28.445624 11433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:40103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:28.445822 11433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:28.446075 11433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:27:28.452320 11433 server_base.cc:1034] running on GCE node
W20260619 08:27:28.452315 11439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:28.452338 11441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:28.452315 11438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:28.452749 11433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:28.453011 11433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:28.454159 11433 hybrid_clock.cc:648] HybridClock initialized: now 1781857648454146 us; error 31 us; skew 500 ppm
I20260619 08:27:28.455569 11433 webserver.cc:460] Webserver started at http://127.25.127.130:44213/ using document root <none> and password file <none>
I20260619 08:27:28.455821 11433 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:28.455865 11433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:28.455961 11433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:28.457124 11433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "b5f08332c14a4fe5bd01a8084542897d"
format_stamp: "Formatted at 2026-06-19 08:27:28 on dist-test-slave-5rhg"
server_key: "baac3d769a80074e852fd977b9451524"
server_key_iv: "110de4ade04c1ace8d960c588ddeb200"
server_key_version: "encryptionkey@0"
I20260619 08:27:28.457429 11433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "b5f08332c14a4fe5bd01a8084542897d"
format_stamp: "Formatted at 2026-06-19 08:27:28 on dist-test-slave-5rhg"
server_key: "baac3d769a80074e852fd977b9451524"
server_key_iv: "110de4ade04c1ace8d960c588ddeb200"
server_key_version: "encryptionkey@0"
I20260619 08:27:28.458750 11433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:27:28.459497 11447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:28.459681 11433 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:28.459791 11433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "b5f08332c14a4fe5bd01a8084542897d"
format_stamp: "Formatted at 2026-06-19 08:27:28 on dist-test-slave-5rhg"
server_key: "baac3d769a80074e852fd977b9451524"
server_key_iv: "110de4ade04c1ace8d960c588ddeb200"
server_key_version: "encryptionkey@0"
I20260619 08:27:28.459875 11433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:28.471206 11433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:28.471539 11433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:28.471702 11433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:28.471942 11433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:28.472386 11433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:28.472443 11433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:28.472487 11433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:28.472514 11433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:28.478446 11433 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:35841
I20260619 08:27:28.478495 11561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:35841 every 8 connection(s)
I20260619 08:27:28.478827 11433 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:27:28.484112 11280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.479572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:57709 (local address 127.25.127.190:40103)
0619 08:27:28.479661 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:28.479663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:28.479938 (+ 275us) server_negotiation.cc:407] Connection header received
0619 08:27:28.480560 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:28.480562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:28.480572 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:28.480630 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:28.481311 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.481812 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.482827 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.483029 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.483091 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.483897 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:28.483898 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:28.483899 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:28.483903 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:28.483919 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:28.483922 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:28.483966 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:28.484026 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 08:27:28.484048 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 08:27:28.484407 11564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.479582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:40103 (local address 127.25.127.130:57709)
0619 08:27:28.479809 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:28.479843 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:28.480454 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:28.480686 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:28.480690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:28.480699 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:28.481219 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:28.481224 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.481897 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.481898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:28.482730 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:28.482732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.483113 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.483114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:28.483245 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.483762 (+ 517us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:27:28.483762 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:28.483791 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:27:28.483815 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:28.483966 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:28.483968 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:28.483970 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:27:28.484017 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 08:27:28.484096 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":95,"threads_started":1}
I20260619 08:27:28.484874 11562 heartbeater.cc:347] Connected to a master server at 127.25.127.190:40103
I20260619 08:27:28.484993 11562 heartbeater.cc:464] Registering TS with master...
I20260619 08:27:28.485193 11562 heartbeater.cc:511] Master 127.25.127.190:40103 requested a full tablet report, sending...
I20260619 08:27:28.485605 11241 ts_manager.cc:194] Registered new tserver with Master: b5f08332c14a4fe5bd01a8084542897d (127.25.127.130:35841)
I20260619 08:27:28.485723 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 11433
I20260619 08:27:28.485849 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:27:28.486050 26110 external_mini_cluster.cc:1428] Setting key 9086175cb0aa2d64af05f35d936f3f0e
I20260619 08:27:28.486090 11241 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.127.130:57709
I20260619 08:27:28.487008 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:40103
--builtin_ntp_servers=127.25.127.148:42455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:27:28.570453 11565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:28.570721 11565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:28.570762 11565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:28.571983 11565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:28.572065 11565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:28.572113 11565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:27:28.573555 11565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:40103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:28.573851 11565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:28.574211 11565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:28.579803 11570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:28.579809 11571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:28.580017 11565 server_base.cc:1034] running on GCE node
W20260619 08:27:28.579803 11573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:28.580443 11565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:28.580726 11565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:28.581907 11565 hybrid_clock.cc:648] HybridClock initialized: now 1781857648581889 us; error 29 us; skew 500 ppm
I20260619 08:27:28.583204 11565 webserver.cc:460] Webserver started at http://127.25.127.131:46657/ using document root <none> and password file <none>
I20260619 08:27:28.583474 11565 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:28.583544 11565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:28.583703 11565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:28.584893 11565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "a113dbee1d594b8589c17d9e0b9cbb21"
format_stamp: "Formatted at 2026-06-19 08:27:28 on dist-test-slave-5rhg"
server_key: "0a61929a7988ca94a1e7b592f1970f96"
server_key_iv: "71bc1479fd222111a8fff33517bb83f4"
server_key_version: "encryptionkey@0"
I20260619 08:27:28.585289 11565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "a113dbee1d594b8589c17d9e0b9cbb21"
format_stamp: "Formatted at 2026-06-19 08:27:28 on dist-test-slave-5rhg"
server_key: "0a61929a7988ca94a1e7b592f1970f96"
server_key_iv: "71bc1479fd222111a8fff33517bb83f4"
server_key_version: "encryptionkey@0"
I20260619 08:27:28.586683 11565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:27:28.587467 11579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:28.587644 11565 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:27:28.587754 11565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "a113dbee1d594b8589c17d9e0b9cbb21"
format_stamp: "Formatted at 2026-06-19 08:27:28 on dist-test-slave-5rhg"
server_key: "0a61929a7988ca94a1e7b592f1970f96"
server_key_iv: "71bc1479fd222111a8fff33517bb83f4"
server_key_version: "encryptionkey@0"
I20260619 08:27:28.587836 11565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:28.605937 11565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:28.606310 11565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:28.606480 11565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:28.606731 11565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:28.607081 11565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:28.607146 11565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:28.607189 11565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:28.607213 11565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:28.613076 11565 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:42193
I20260619 08:27:28.613117 11693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:42193 every 8 connection(s)
I20260619 08:27:28.613456 11565 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:27:28.619019 11280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.614211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:50683 (local address 127.25.127.190:40103)
0619 08:27:28.614283 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:28.614285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:28.614516 (+ 231us) server_negotiation.cc:407] Connection header received
0619 08:27:28.615076 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:28.615077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:28.615089 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:28.615200 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:28.615933 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.616507 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.617525 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.617723 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.617800 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.618697 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:28.618698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:28.618699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:28.618705 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:28.618720 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:28.618729 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:28.618810 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:28.618906 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 08:27:28.618933 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 08:27:28.619282 11696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.614214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:40103 (local address 127.25.127.131:50683)
0619 08:27:28.614398 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:28.614432 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:28.614967 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:28.615290 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:28.615295 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:28.615303 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:28.615821 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:28.615826 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.616610 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.616611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:28.617418 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:28.617420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.617822 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.617824 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:28.617961 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.618516 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:27:28.618517 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:28.618544 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:27:28.618567 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:28.618811 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:28.618819 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:28.618822 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:27:28.618909 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 08:27:28.619016 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260619 08:27:28.619779 11694 heartbeater.cc:347] Connected to a master server at 127.25.127.190:40103
I20260619 08:27:28.619895 11694 heartbeater.cc:464] Registering TS with master...
I20260619 08:27:28.620100 11694 heartbeater.cc:511] Master 127.25.127.190:40103 requested a full tablet report, sending...
I20260619 08:27:28.620532 11241 ts_manager.cc:194] Registered new tserver with Master: a113dbee1d594b8589c17d9e0b9cbb21 (127.25.127.131:42193)
I20260619 08:27:28.620889 11241 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.127.131:50683
I20260619 08:27:28.621914 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 11565
I20260619 08:27:28.621994 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:27:28.622215 26110 external_mini_cluster.cc:1428] Setting key 204bb8b053a2e0be8bcd9fb8dbbd25bc
I20260619 08:27:28.623116 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:27:28.627182 11280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.624570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32838 (local address 127.25.127.190:40103)
0619 08:27:28.624642 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:28.624644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:28.624721 (+ 77us) server_negotiation.cc:407] Connection header received
0619 08:27:28.624776 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:28.624777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:28.624785 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:28.624833 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:28.624985 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.625462 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.626175 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.626346 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.626396 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.626978 (+ 582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:28.626979 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:28.626979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:28.626983 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:28.626995 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:28.626999 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:28.627040 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:28.627086 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 08:27:28.627104 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 08:27:28.628654 11241 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:27:28.630183 11241 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:27:28.636725 11704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.632368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:42193 (local address 127.0.0.1:59436)
0619 08:27:28.632540 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:28.632550 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:28.632593 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:28.633016 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:28.633019 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:28.633021 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:28.633090 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:28.633092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.634151 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.634152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:28.635450 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:28.635453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.636405 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.636406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:28.636491 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.636510 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 08:27:28.636580 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 08:27:28.636641 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":83,"threads_started":1}
I20260619 08:27:28.636739 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.632620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59436 (local address 127.25.127.131:42193)
0619 08:27:28.632856 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:28.632860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:28.632870 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:27:28.632912 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:28.632913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:28.632925 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:28.633012 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:28.633238 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.634052 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.635534 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.636315 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.636393 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.636484 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:28.636592 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 08:27:28.636638 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":110,"threads_started":1}
I20260619 08:27:28.636953 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.632360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:43057 (local address 127.0.0.1:36924)
0619 08:27:28.632482 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:28.632525 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:28.632593 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:28.633281 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:28.633282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:28.633285 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:28.633346 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:28.633347 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.634682 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.634683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:28.635763 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:28.635765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.636696 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.636697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:28.636761 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.636766 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:27:28.636796 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 08:27:28.636850 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":25,"thread_start_us":75,"threads_started":1}
I20260619 08:27:28.637043 11707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.632891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54964 (local address 127.25.127.130:35841)
0619 08:27:28.633061 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:28.633066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:28.633079 (+ 13us) server_negotiation.cc:407] Connection header received
0619 08:27:28.633126 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:28.633128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:28.633144 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:28.633241 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:28.633446 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.634590 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.635741 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.636448 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.636519 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.636608 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:28.636902 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 08:27:28.636955 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260619 08:27:28.637442 11703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.632368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:35841 (local address 127.0.0.1:54964)
0619 08:27:28.632690 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:28.632698 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:28.632772 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:28.633309 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:28.633309 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:28.633311 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:28.633352 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:28.633353 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.634693 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.634694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:28.635646 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:28.635648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.636653 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.636654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:28.636708 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.636711 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:27:28.636760 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 08:27:28.636805 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":3,"thread_start_us":84,"threads_started":1}
I20260619 08:27:28.637538 11628 tablet_service.cc:1467] Processing CreateTablet for tablet eabd1ac80a2647f1800b9b98d092249a (DEFAULT_TABLE table=test-table [id=e7d9cdabd3e64d25aefe3ad2b20498ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:27:28.637810 11706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.632637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36924 (local address 127.25.127.129:43057)
0619 08:27:28.633115 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:28.633119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:28.633128 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:27:28.633164 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:28.633167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:28.633182 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:28.633438 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:28.633503 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.634618 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.635861 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.636601 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.637559 (+ 958us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.637657 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:28.637682 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 08:27:28.637734 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":114,"threads_started":1}
I20260619 08:27:28.637944 11628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eabd1ac80a2647f1800b9b98d092249a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:28.638381 11363 tablet_service.cc:1467] Processing CreateTablet for tablet eabd1ac80a2647f1800b9b98d092249a (DEFAULT_TABLE table=test-table [id=e7d9cdabd3e64d25aefe3ad2b20498ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:27:28.638682 11363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eabd1ac80a2647f1800b9b98d092249a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:28.640591 11496 tablet_service.cc:1467] Processing CreateTablet for tablet eabd1ac80a2647f1800b9b98d092249a (DEFAULT_TABLE table=test-table [id=e7d9cdabd3e64d25aefe3ad2b20498ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:27:28.640897 11708 tablet_bootstrap.cc:492] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21: Bootstrap starting.
I20260619 08:27:28.640965 11496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eabd1ac80a2647f1800b9b98d092249a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:28.641574 11709 tablet_bootstrap.cc:492] T eabd1ac80a2647f1800b9b98d092249a P 8e7a590329ce4c4aa7368d9d438c2464: Bootstrap starting.
I20260619 08:27:28.641757 11708 tablet_bootstrap.cc:654] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:28.642064 11708 log.cc:824] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:28.642227 11709 tablet_bootstrap.cc:654] T eabd1ac80a2647f1800b9b98d092249a P 8e7a590329ce4c4aa7368d9d438c2464: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:28.642551 11709 log.cc:824] T eabd1ac80a2647f1800b9b98d092249a P 8e7a590329ce4c4aa7368d9d438c2464: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:28.642858 11708 tablet_bootstrap.cc:492] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21: No bootstrap required, opened a new log
I20260619 08:27:28.642936 11708 ts_tablet_manager.cc:1397] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:27:28.643325 11709 tablet_bootstrap.cc:492] T eabd1ac80a2647f1800b9b98d092249a P 8e7a590329ce4c4aa7368d9d438c2464: No bootstrap required, opened a new log
I20260619 08:27:28.643431 11709 ts_tablet_manager.cc:1397] T eabd1ac80a2647f1800b9b98d092249a P 8e7a590329ce4c4aa7368d9d438c2464: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:27:28.644378 11712 tablet_bootstrap.cc:492] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d: Bootstrap starting.
I20260619 08:27:28.644990 11712 tablet_bootstrap.cc:654] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:28.645315 11712 log.cc:824] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:28.645744 11708 raft_consensus.cc:348] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7a590329ce4c4aa7368d9d438c2464" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43057 } } peers { permanent_uuid: "b5f08332c14a4fe5bd01a8084542897d" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35841 } } peers { permanent_uuid: "a113dbee1d594b8589c17d9e0b9cbb21" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42193 } }
I20260619 08:27:28.645943 11708 raft_consensus.cc:374] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:28.645980 11708 raft_consensus.cc:729] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a113dbee1d594b8589c17d9e0b9cbb21, State: Initialized, Role: FOLLOWER
I20260619 08:27:28.645870 11709 raft_consensus.cc:348] T eabd1ac80a2647f1800b9b98d092249a P 8e7a590329ce4c4aa7368d9d438c2464 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7a590329ce4c4aa7368d9d438c2464" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43057 } } peers { permanent_uuid: "b5f08332c14a4fe5bd01a8084542897d" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35841 } } peers { permanent_uuid: "a113dbee1d594b8589c17d9e0b9cbb21" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42193 } }
I20260619 08:27:28.646027 11709 raft_consensus.cc:374] T eabd1ac80a2647f1800b9b98d092249a P 8e7a590329ce4c4aa7368d9d438c2464 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:28.646060 11709 raft_consensus.cc:729] T eabd1ac80a2647f1800b9b98d092249a P 8e7a590329ce4c4aa7368d9d438c2464 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e7a590329ce4c4aa7368d9d438c2464, State: Initialized, Role: FOLLOWER
I20260619 08:27:28.646085 11712 tablet_bootstrap.cc:492] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d: No bootstrap required, opened a new log
I20260619 08:27:28.646137 11712 ts_tablet_manager.cc:1397] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:27:28.646124 11708 consensus_queue.cc:260] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7a590329ce4c4aa7368d9d438c2464" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43057 } } peers { permanent_uuid: "b5f08332c14a4fe5bd01a8084542897d" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35841 } } peers { permanent_uuid: "a113dbee1d594b8589c17d9e0b9cbb21" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42193 } }
I20260619 08:27:28.646188 11709 consensus_queue.cc:260] T eabd1ac80a2647f1800b9b98d092249a P 8e7a590329ce4c4aa7368d9d438c2464 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7a590329ce4c4aa7368d9d438c2464" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43057 } } peers { permanent_uuid: "b5f08332c14a4fe5bd01a8084542897d" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35841 } } peers { permanent_uuid: "a113dbee1d594b8589c17d9e0b9cbb21" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42193 } }
I20260619 08:27:28.646396 11708 ts_tablet_manager.cc:1428] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260619 08:27:28.646456 11709 ts_tablet_manager.cc:1428] T eabd1ac80a2647f1800b9b98d092249a P 8e7a590329ce4c4aa7368d9d438c2464: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:27:28.646474 11694 heartbeater.cc:503] Master 127.25.127.190:40103 was elected leader, sending a full tablet report...
I20260619 08:27:28.646595 11430 heartbeater.cc:503] Master 127.25.127.190:40103 was elected leader, sending a full tablet report...
I20260619 08:27:28.648725 11712 raft_consensus.cc:348] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7a590329ce4c4aa7368d9d438c2464" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43057 } } peers { permanent_uuid: "b5f08332c14a4fe5bd01a8084542897d" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35841 } } peers { permanent_uuid: "a113dbee1d594b8589c17d9e0b9cbb21" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42193 } }
I20260619 08:27:28.648860 11712 raft_consensus.cc:374] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:28.648882 11712 raft_consensus.cc:729] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f08332c14a4fe5bd01a8084542897d, State: Initialized, Role: FOLLOWER
I20260619 08:27:28.648954 11712 consensus_queue.cc:260] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7a590329ce4c4aa7368d9d438c2464" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43057 } } peers { permanent_uuid: "b5f08332c14a4fe5bd01a8084542897d" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35841 } } peers { permanent_uuid: "a113dbee1d594b8589c17d9e0b9cbb21" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42193 } }
I20260619 08:27:28.649130 11712 ts_tablet_manager.cc:1428] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:27:28.649266 11562 heartbeater.cc:503] Master 127.25.127.190:40103 was elected leader, sending a full tablet report...
I20260619 08:27:28.704751 11714 raft_consensus.cc:482] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:27:28.704958 11714 raft_consensus.cc:504] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7a590329ce4c4aa7368d9d438c2464" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43057 } } peers { permanent_uuid: "b5f08332c14a4fe5bd01a8084542897d" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35841 } } peers { permanent_uuid: "a113dbee1d594b8589c17d9e0b9cbb21" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42193 } }
I20260619 08:27:28.705492 11714 leader_election.cc:288] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e7a590329ce4c4aa7368d9d438c2464 (127.25.127.129:43057), b5f08332c14a4fe5bd01a8084542897d (127.25.127.130:35841)
I20260619 08:27:28.708441 11706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.705683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:48111 (local address 127.25.127.129:43057)
0619 08:27:28.705731 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:28.705733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:28.705742 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:27:28.705819 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:28.705820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:28.705831 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:28.705879 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:28.706049 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.706591 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.707563 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.708042 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.708127 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.708136 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:28.708332 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 08:27:28.708374 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:27:28.708439 11696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.705623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:43057 (local address 127.25.127.131:48111)
0619 08:27:28.705693 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:28.705705 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:28.705749 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:28.705891 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:28.705892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:28.705893 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:28.705969 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:28.705973 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.706682 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.706683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:28.707463 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:28.707465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.708124 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.708125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:28.708237 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.708240 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:27:28.708323 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 08:27:28.708365 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 08:27:28.708521 11707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.705797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:39511 (local address 127.25.127.130:35841)
0619 08:27:28.705845 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:28.705847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:28.705985 (+ 138us) server_negotiation.cc:407] Connection header received
0619 08:27:28.706058 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:28.706060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:28.706070 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:28.706163 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:28.706334 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.706828 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.707760 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.708123 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.708235 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.708243 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:28.708408 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 08:27:28.708448 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:27:28.708547 11717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.705728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:35841 (local address 127.25.127.131:39511)
0619 08:27:28.705900 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:28.705909 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:28.705995 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:28.706174 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:28.706175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:28.706176 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:28.706252 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:28.706253 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.706917 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.706919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:28.707664 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:28.707665 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.708258 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.708260 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:28.708329 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.708337 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 08:27:28.708402 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 08:27:28.708442 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":93,"threads_started":1}
I20260619 08:27:28.708868 11516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eabd1ac80a2647f1800b9b98d092249a" candidate_uuid: "a113dbee1d594b8589c17d9e0b9cbb21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f08332c14a4fe5bd01a8084542897d" is_pre_election: true
I20260619 08:27:28.708956 11384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eabd1ac80a2647f1800b9b98d092249a" candidate_uuid: "a113dbee1d594b8589c17d9e0b9cbb21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e7a590329ce4c4aa7368d9d438c2464" is_pre_election: true
I20260619 08:27:28.709040 11516 raft_consensus.cc:2454] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a113dbee1d594b8589c17d9e0b9cbb21 in term 0.
I20260619 08:27:28.709088 11384 raft_consensus.cc:2454] T eabd1ac80a2647f1800b9b98d092249a P 8e7a590329ce4c4aa7368d9d438c2464 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a113dbee1d594b8589c17d9e0b9cbb21 in term 0.
I20260619 08:27:28.709302 11581 leader_election.cc:302] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a113dbee1d594b8589c17d9e0b9cbb21, b5f08332c14a4fe5bd01a8084542897d; no voters:
I20260619 08:27:28.709489 11714 raft_consensus.cc:2781] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:27:28.709530 11714 raft_consensus.cc:482] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:27:28.709546 11714 raft_consensus.cc:3037] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:28.710234 11714 raft_consensus.cc:504] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7a590329ce4c4aa7368d9d438c2464" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43057 } } peers { permanent_uuid: "b5f08332c14a4fe5bd01a8084542897d" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35841 } } peers { permanent_uuid: "a113dbee1d594b8589c17d9e0b9cbb21" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42193 } }
I20260619 08:27:28.710382 11714 leader_election.cc:288] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [CANDIDATE]: Term 1 election: Requested vote from peers 8e7a590329ce4c4aa7368d9d438c2464 (127.25.127.129:43057), b5f08332c14a4fe5bd01a8084542897d (127.25.127.130:35841)
I20260619 08:27:28.710532 11384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eabd1ac80a2647f1800b9b98d092249a" candidate_uuid: "a113dbee1d594b8589c17d9e0b9cbb21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e7a590329ce4c4aa7368d9d438c2464"
I20260619 08:27:28.710553 11516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eabd1ac80a2647f1800b9b98d092249a" candidate_uuid: "a113dbee1d594b8589c17d9e0b9cbb21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f08332c14a4fe5bd01a8084542897d"
I20260619 08:27:28.710628 11384 raft_consensus.cc:3037] T eabd1ac80a2647f1800b9b98d092249a P 8e7a590329ce4c4aa7368d9d438c2464 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:28.710628 11516 raft_consensus.cc:3037] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:28.711562 11516 raft_consensus.cc:2454] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a113dbee1d594b8589c17d9e0b9cbb21 in term 1.
I20260619 08:27:28.711585 11384 raft_consensus.cc:2454] T eabd1ac80a2647f1800b9b98d092249a P 8e7a590329ce4c4aa7368d9d438c2464 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a113dbee1d594b8589c17d9e0b9cbb21 in term 1.
I20260619 08:27:28.711752 11580 leader_election.cc:302] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e7a590329ce4c4aa7368d9d438c2464, a113dbee1d594b8589c17d9e0b9cbb21; no voters:
I20260619 08:27:28.711851 11714 raft_consensus.cc:2781] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:28.711988 11714 raft_consensus.cc:686] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [term 1 LEADER]: Becoming Leader. State: Replica: a113dbee1d594b8589c17d9e0b9cbb21, State: Running, Role: LEADER
I20260619 08:27:28.712090 11714 consensus_queue.cc:237] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7a590329ce4c4aa7368d9d438c2464" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43057 } } peers { permanent_uuid: "b5f08332c14a4fe5bd01a8084542897d" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35841 } } peers { permanent_uuid: "a113dbee1d594b8589c17d9e0b9cbb21" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42193 } }
I20260619 08:27:28.712988 11241 catalog_manager.cc:5495] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 reported cstate change: term changed from 0 to 1, leader changed from <none> to a113dbee1d594b8589c17d9e0b9cbb21 (127.25.127.131). New cstate: current_term: 1 leader_uuid: "a113dbee1d594b8589c17d9e0b9cbb21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7a590329ce4c4aa7368d9d438c2464" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f08332c14a4fe5bd01a8084542897d" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a113dbee1d594b8589c17d9e0b9cbb21" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 42193 } health_report { overall_health: HEALTHY } } }
I20260619 08:27:28.728278 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.725882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59438 (local address 127.25.127.131:42193)
0619 08:27:28.725928 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:28.725930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:28.725938 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:27:28.725991 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:28.725992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:28.726009 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:28.726085 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:27:28.726221 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.726679 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.727452 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:28.727646 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:28.727754 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:28.727897 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:27:28.728003 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:27:28.728100 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:28.728118 (+ 18us) server_negotiation.cc:299] Negotiation successful
0619 08:27:28.728137 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 08:27:28.729413 11563 tablet.cc:2369] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:27:28.730192 11516 raft_consensus.cc:1261] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d [term 1 FOLLOWER]: Refusing update from remote peer a113dbee1d594b8589c17d9e0b9cbb21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:27:28.730183 11384 raft_consensus.cc:1261] T eabd1ac80a2647f1800b9b98d092249a P 8e7a590329ce4c4aa7368d9d438c2464 [term 1 FOLLOWER]: Refusing update from remote peer a113dbee1d594b8589c17d9e0b9cbb21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:27:28.730553 11714 consensus_queue.cc:1035] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f08332c14a4fe5bd01a8084542897d" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:27:28.730720 11718 consensus_queue.cc:1035] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e7a590329ce4c4aa7368d9d438c2464" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 43057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:27:28.733454 11721 mvcc.cc:205] Tried to move back new op lower bound from 7298488929195761664 to 7298488929125675008. Current Snapshot: MvccSnapshot[applied={T|T < 7298488929195761664}]
I20260619 08:27:28.733480 11723 mvcc.cc:205] Tried to move back new op lower bound from 7298488929195761664 to 7298488929125675008. Current Snapshot: MvccSnapshot[applied={T|T < 7298488929195761664}]
I20260619 08:27:28.735574 11241 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:32838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:27:28.735683 11241 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:32838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:27:28.736495 11241 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 899551f0617e4a33bf7d536794815643: Sending DeleteTablet for 3 replicas of tablet eabd1ac80a2647f1800b9b98d092249a
I20260619 08:27:28.736887 11496 tablet_service.cc:1514] Processing DeleteTablet for tablet eabd1ac80a2647f1800b9b98d092249a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:27:28 UTC) from {username='slave'} at 127.0.0.1:54964
I20260619 08:27:28.736913 11363 tablet_service.cc:1514] Processing DeleteTablet for tablet eabd1ac80a2647f1800b9b98d092249a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:27:28 UTC) from {username='slave'} at 127.0.0.1:36924
I20260619 08:27:28.736940 11628 tablet_service.cc:1514] Processing DeleteTablet for tablet eabd1ac80a2647f1800b9b98d092249a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:27:28 UTC) from {username='slave'} at 127.0.0.1:59436
I20260619 08:27:28.737150 11732 tablet_replica.cc:330] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21: stopping tablet replica
I20260619 08:27:28.737248 11732 raft_consensus.cc:2229] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:27:28.737351 11732 raft_consensus.cc:2258] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:27:28.737452 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 11301
I20260619 08:27:28.737818 11732 ts_tablet_manager.cc:1905] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:27:28.739490 11732 ts_tablet_manager.cc:1918] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:27:28.739573 11732 log.cc:1196] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/eabd1ac80a2647f1800b9b98d092249a
I20260619 08:27:28.739859 11732 ts_tablet_manager.cc:1939] T eabd1ac80a2647f1800b9b98d092249a P a113dbee1d594b8589c17d9e0b9cbb21: Deleting consensus metadata
I20260619 08:27:28.740337 11731 tablet_replica.cc:330] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d: stopping tablet replica
I20260619 08:27:28.740471 11731 raft_consensus.cc:2229] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:27:28.740577 11731 raft_consensus.cc:2258] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:27:28.742301 11227 catalog_manager.cc:4841] TS a113dbee1d594b8589c17d9e0b9cbb21 (127.25.127.131:42193): tablet eabd1ac80a2647f1800b9b98d092249a (table test-table [id=e7d9cdabd3e64d25aefe3ad2b20498ed]) successfully deleted
I20260619 08:27:28.742329 11731 ts_tablet_manager.cc:1905] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:27:28.743999 11731 ts_tablet_manager.cc:1918] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:27:28.744086 11731 log.cc:1196] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/eabd1ac80a2647f1800b9b98d092249a
I20260619 08:27:28.744397 11731 ts_tablet_manager.cc:1939] T eabd1ac80a2647f1800b9b98d092249a P b5f08332c14a4fe5bd01a8084542897d: Deleting consensus metadata
I20260619 08:27:28.744861 11226 catalog_manager.cc:4841] TS b5f08332c14a4fe5bd01a8084542897d (127.25.127.130:35841): tablet eabd1ac80a2647f1800b9b98d092249a (table test-table [id=e7d9cdabd3e64d25aefe3ad2b20498ed]) successfully deleted
W20260619 08:27:28.746163 11225 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.127.129:43057 (error 108)
I20260619 08:27:28.746470 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 11433
I20260619 08:27:28.747228 11703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:28.746539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:43057 (local address 127.0.0.1:36926)
0619 08:27:28.747101 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:28.747194 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.129:43057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":533}
W20260619 08:27:28.748466 11225 catalog_manager.cc:4584] TS 8e7a590329ce4c4aa7368d9d438c2464 (127.25.127.129:43057): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eabd1ac80a2647f1800b9b98d092249a: Network error: Client connection negotiation failed: client connection to 127.25.127.129:43057: connect: Connection refused (error 111)
I20260619 08:27:28.753082 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 11565
I20260619 08:27:28.757441 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 11210
2026-06-19T08:27:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (740 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:28 dist-test-slave-5rhg krb5kdc[11738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:28 dist-test-slave-5rhg krb5kdc[11738](info): set up 2 sockets
Jun 19 08:27:28 dist-test-slave-5rhg krb5kdc[11738](info): commencing operation
krb5kdc: starting...
W20260619 08:27:29.792829 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:29 dist-test-slave-5rhg krb5kdc[11738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:27:29.910612 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:40447
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:42941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:40447
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:29.993104 11751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:29.993369 11751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:29.993409 11751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:29.994614 11751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:29.994686 11751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:29.994712 11751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:29.994743 11751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:29.994776 11751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:29.996649 11751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:40447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:40447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:29.996954 11751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:29.997264 11751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:30.002266 11756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:30.002326 11757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:30.002338 11759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:30.002616 11751 server_base.cc:1034] running on GCE node
I20260619 08:27:30.002861 11751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:30.003188 11751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:30.004372 11751 hybrid_clock.cc:648] HybridClock initialized: now 1781857650004355 us; error 44 us; skew 500 ppm
Jun 19 08:27:30 dist-test-slave-5rhg krb5kdc[11738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857650, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:30.006958 11751 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:27:30.007520 11751 webserver.cc:460] Webserver started at http://127.25.127.190:45197/ using document root <none> and password file <none>
I20260619 08:27:30.007786 11751 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:30.007858 11751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:30.008018 11751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:30.009235 11751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "ca62df64bcb844acbeb5e7d6d3957a1e"
format_stamp: "Formatted at 2026-06-19 08:27:30 on dist-test-slave-5rhg"
server_key: "c32f4f4522d94673323ba28cbbaabdd6"
server_key_iv: "5759a7027411d5fd6868b31da0e831f6"
server_key_version: "encryptionkey@0"
I20260619 08:27:30.009626 11751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "ca62df64bcb844acbeb5e7d6d3957a1e"
format_stamp: "Formatted at 2026-06-19 08:27:30 on dist-test-slave-5rhg"
server_key: "c32f4f4522d94673323ba28cbbaabdd6"
server_key_iv: "5759a7027411d5fd6868b31da0e831f6"
server_key_version: "encryptionkey@0"
I20260619 08:27:30.011027 11751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:27:30.011807 11766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:30.012005 11751 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:30.012123 11751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "ca62df64bcb844acbeb5e7d6d3957a1e"
format_stamp: "Formatted at 2026-06-19 08:27:30 on dist-test-slave-5rhg"
server_key: "c32f4f4522d94673323ba28cbbaabdd6"
server_key_iv: "5759a7027411d5fd6868b31da0e831f6"
server_key_version: "encryptionkey@0"
I20260619 08:27:30.012257 11751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:30.031617 11751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:30.034165 11751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:30.034345 11751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:30.038651 11751 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:40447
I20260619 08:27:30.038700 11818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:40447 every 8 connection(s)
I20260619 08:27:30.039081 11751 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:30.039948 11819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:30.042342 11819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e: Bootstrap starting.
I20260619 08:27:30.042963 11819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:30.043277 11819 log.cc:824] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:30.044006 11819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e: No bootstrap required, opened a new log
I20260619 08:27:30.045717 11819 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca62df64bcb844acbeb5e7d6d3957a1e" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40447 } }
I20260619 08:27:30.045868 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 11751
I20260619 08:27:30.045917 11819 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:30.045975 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:30.045961 11819 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca62df64bcb844acbeb5e7d6d3957a1e, State: Initialized, Role: FOLLOWER
I20260619 08:27:30.046165 26110 external_mini_cluster.cc:1428] Setting key e905656f08f36c59181188a6918097fc
I20260619 08:27:30.046116 11819 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca62df64bcb844acbeb5e7d6d3957a1e" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40447 } }
I20260619 08:27:30.046209 11819 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:30.046244 11819 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:30.046299 11819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:30.047083 11819 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca62df64bcb844acbeb5e7d6d3957a1e" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40447 } }
I20260619 08:27:30.047212 11819 leader_election.cc:302] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca62df64bcb844acbeb5e7d6d3957a1e; no voters:
I20260619 08:27:30.047381 11819 leader_election.cc:288] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:30.047452 11824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:30.047631 11824 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e [term 1 LEADER]: Becoming Leader. State: Replica: ca62df64bcb844acbeb5e7d6d3957a1e, State: Running, Role: LEADER
I20260619 08:27:30.047636 11819 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:30.047781 11824 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca62df64bcb844acbeb5e7d6d3957a1e" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40447 } }
I20260619 08:27:30.048163 11825 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca62df64bcb844acbeb5e7d6d3957a1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca62df64bcb844acbeb5e7d6d3957a1e" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40447 } } }
I20260619 08:27:30.048326 11825 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:30.048163 11824 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca62df64bcb844acbeb5e7d6d3957a1e. Latest consensus state: current_term: 1 leader_uuid: "ca62df64bcb844acbeb5e7d6d3957a1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca62df64bcb844acbeb5e7d6d3957a1e" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40447 } } }
I20260619 08:27:30.048832 11824 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e [sys.catalog]: This master's current role is: LEADER
W20260619 08:27:30.049626 11840 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:27:30.049685 11840 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:27:30.049783 11833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:30.049970 11833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:30.051478 11833 catalog_manager.cc:1347] Generated new cluster ID: 41d2f3d493394e209bfab407ecfe6c95
I20260619 08:27:30.051528 11833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:27:30 dist-test-slave-5rhg krb5kdc[11738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:27:30.058909 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.046702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40894 (local address 127.25.127.190:40447)
0619 08:27:30.047282 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:30.047291 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:30.047331 (+ 40us) server_negotiation.cc:407] Connection header received
0619 08:27:30.047459 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:30.047468 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:30.047531 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:30.048627 (+ 1096us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:30.048766 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.050295 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.051354 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.052010 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.052447 (+ 437us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.054909 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:30.054914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:30.054914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:30.054948 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:30.057004 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:30.057381 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:30.057384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:30.057384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:30.057506 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:30.057697 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:30.057698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:30.057698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:30.058000 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:30.058480 (+ 480us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:30.058552 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 08:27:30.058696 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":488,"threads_started":1}
I20260619 08:27:30.063031 11833 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:30.063582 11833 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:30.071372 11833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e: Generated new TSK 0
I20260619 08:27:30.071602 11833 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
I20260619 08:27:30.103188 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:40447
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:42941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:30.204890 11845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:30.205098 11845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:30.205123 11845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:30.206322 11845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:30.206367 11845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:30.206395 11845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:27:30.207948 11845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:40447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:30.208235 11845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:30.208487 11845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:30.214074 11851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:30.214082 11853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:30.214089 11850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:30.214310 11845 server_base.cc:1034] running on GCE node
I20260619 08:27:30.214516 11845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:30.214773 11845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:30.215914 11845 hybrid_clock.cc:648] HybridClock initialized: now 1781857650215906 us; error 47 us; skew 500 ppm
Jun 19 08:27:30 dist-test-slave-5rhg krb5kdc[11738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857650, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:30.218576 11845 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:27:30.219022 11845 webserver.cc:460] Webserver started at http://127.25.127.129:42839/ using document root <none> and password file <none>
I20260619 08:27:30.219273 11845 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:30.219341 11845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:30.219491 11845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:30.220633 11845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "4b86528bbd214985b69d2e0257740075"
format_stamp: "Formatted at 2026-06-19 08:27:30 on dist-test-slave-5rhg"
server_key: "988082f64c648c039824cb7de083eb97"
server_key_iv: "506c208a7ab9f0997e21489bf2389f82"
server_key_version: "encryptionkey@0"
I20260619 08:27:30.220989 11845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "4b86528bbd214985b69d2e0257740075"
format_stamp: "Formatted at 2026-06-19 08:27:30 on dist-test-slave-5rhg"
server_key: "988082f64c648c039824cb7de083eb97"
server_key_iv: "506c208a7ab9f0997e21489bf2389f82"
server_key_version: "encryptionkey@0"
I20260619 08:27:30.222335 11845 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:30.223148 11860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:30.223357 11845 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:30.223465 11845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "4b86528bbd214985b69d2e0257740075"
format_stamp: "Formatted at 2026-06-19 08:27:30 on dist-test-slave-5rhg"
server_key: "988082f64c648c039824cb7de083eb97"
server_key_iv: "506c208a7ab9f0997e21489bf2389f82"
server_key_version: "encryptionkey@0"
I20260619 08:27:30.223553 11845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:30.240008 11845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:30.241955 11845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:30.242136 11845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:30.242391 11845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:30.242723 11845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:30.242779 11845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:30.242820 11845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:30.242869 11845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:30.248989 11845 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:36813
I20260619 08:27:30.249078 11974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:36813 every 8 connection(s)
I20260619 08:27:30.249454 11845 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:27:30 dist-test-slave-5rhg krb5kdc[11738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857650, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:27:30.259063 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.250240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:48833 (local address 127.25.127.190:40447)
0619 08:27:30.250296 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:30.250299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:30.250542 (+ 243us) server_negotiation.cc:407] Connection header received
0619 08:27:30.251090 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:30.251091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:30.251108 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:30.251166 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:30.252195 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.252713 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.253744 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.253975 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.254063 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.256291 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:30.256293 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:30.256293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:30.256301 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:30.257579 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:30.258018 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:30.258020 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:30.258020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:30.258083 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:30.258344 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:30.258345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:30.258345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:30.258468 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:30.258545 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:30.258842 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 08:27:30.258969 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:27:30.259189 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 11845
I20260619 08:27:30.259261 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.250227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:40447 (local address 127.25.127.129:48833)
0619 08:27:30.250409 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:30.250448 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:30.250978 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:30.251249 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:30.251252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:30.251849 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:30.252061 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:30.252066 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.252826 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.252827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:30.253649 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:30.253651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.254076 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.254077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:30.254215 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.254721 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:27:30.254729 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:30.256132 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:30.257680 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:30.257688 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:30.257691 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:30.257927 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:30.258159 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:30.258159 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:30.258160 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:30.258275 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:30.258535 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:30.258543 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:30.258707 (+ 164us) client_negotiation.cc:769] Sending connection context
0619 08:27:30.258820 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 08:27:30.258993 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260619 08:27:30.259317 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:27:30.259513 26110 external_mini_cluster.cc:1428] Setting key b2aaa8dc664ea629b20ee157caa9c1bd
I20260619 08:27:30.259825 11975 heartbeater.cc:347] Connected to a master server at 127.25.127.190:40447
I20260619 08:27:30.259965 11975 heartbeater.cc:464] Registering TS with master...
I20260619 08:27:30.260226 11975 heartbeater.cc:511] Master 127.25.127.190:40447 requested a full tablet report, sending...
I20260619 08:27:30.260821 11783 ts_manager.cc:194] Registered new tserver with Master: 4b86528bbd214985b69d2e0257740075 (127.25.127.129:36813)
I20260619 08:27:30.261750 11783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:48833
WARNING: no policy specified for kudu/127.25.127.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
I20260619 08:27:30.284719 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:40447
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:42941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:30.367815 11980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:30.368041 11980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:30.368067 11980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:30.369302 11980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:30.369349 11980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:30.369376 11980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:27:30.370827 11980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:40447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:30.371088 11980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:30.371330 11980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:30.376475 11986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:30.376475 11985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:30.376681 11980 server_base.cc:1034] running on GCE node
W20260619 08:27:30.376550 11988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:30.377009 11980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:30.377255 11980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:30.378407 11980 hybrid_clock.cc:648] HybridClock initialized: now 1781857650378386 us; error 38 us; skew 500 ppm
Jun 19 08:27:30 dist-test-slave-5rhg krb5kdc[11738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857650, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:30.381295 11980 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:27:30.381832 11980 webserver.cc:460] Webserver started at http://127.25.127.130:46539/ using document root <none> and password file <none>
I20260619 08:27:30.382077 11980 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:30.382125 11980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:30.382283 11980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:30.383406 11980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "e04dc8fa8f234727be8a65ab3ec81801"
format_stamp: "Formatted at 2026-06-19 08:27:30 on dist-test-slave-5rhg"
server_key: "e7f676c780b7e52e1a7ad26fc4e045c2"
server_key_iv: "54182042a53b9dedbbaf7a9c6330e0a3"
server_key_version: "encryptionkey@0"
I20260619 08:27:30.383782 11980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "e04dc8fa8f234727be8a65ab3ec81801"
format_stamp: "Formatted at 2026-06-19 08:27:30 on dist-test-slave-5rhg"
server_key: "e7f676c780b7e52e1a7ad26fc4e045c2"
server_key_iv: "54182042a53b9dedbbaf7a9c6330e0a3"
server_key_version: "encryptionkey@0"
I20260619 08:27:30.385388 11980 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:27:30.386194 11995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:30.386394 11980 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:30.386487 11980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "e04dc8fa8f234727be8a65ab3ec81801"
format_stamp: "Formatted at 2026-06-19 08:27:30 on dist-test-slave-5rhg"
server_key: "e7f676c780b7e52e1a7ad26fc4e045c2"
server_key_iv: "54182042a53b9dedbbaf7a9c6330e0a3"
server_key_version: "encryptionkey@0"
I20260619 08:27:30.386545 11980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:30.405026 11980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:30.407483 11980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:30.407611 11980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:30.407840 11980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:30.408232 11980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:30.408279 11980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:30.408320 11980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:30.408349 11980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:30.414570 11980 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:38391
I20260619 08:27:30.414651 12109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:38391 every 8 connection(s)
I20260619 08:27:30.415015 11980 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:27:30.419939 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 11980
I20260619 08:27:30.420071 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:27:30.420331 26110 external_mini_cluster.cc:1428] Setting key cddc5cedaa9dcf043050f845eeca6fe8
Jun 19 08:27:30 dist-test-slave-5rhg krb5kdc[11738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857650, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:27:30.425073 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.415883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:40335 (local address 127.25.127.190:40447)
0619 08:27:30.415931 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:30.415933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:30.416175 (+ 242us) server_negotiation.cc:407] Connection header received
0619 08:27:30.416734 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:30.416735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:30.416746 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:30.416837 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:30.417828 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.418332 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.419332 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.419522 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.419581 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.422062 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:30.422064 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:30.422065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:30.422073 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:30.423544 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:30.423986 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:30.423988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:30.423988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:30.424058 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:30.424366 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:30.424367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:30.424367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:30.424496 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:30.424577 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:30.424869 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 08:27:30.424967 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:27:30.425320 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.415871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:40447 (local address 127.25.127.130:40335)
0619 08:27:30.416036 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:30.416074 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:30.416625 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:30.416917 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:30.416920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:30.417505 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:30.417733 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:30.417738 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.418414 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.418415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:30.419222 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:30.419224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.419598 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.419599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:30.419764 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.420486 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:27:30.420497 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:30.421945 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:30.423653 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:30.423661 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:30.423663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:30.423906 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:30.424135 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:30.424136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:30.424136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:30.424311 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:30.424570 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:30.424577 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:30.424729 (+ 152us) client_negotiation.cc:769] Sending connection context
0619 08:27:30.424836 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 08:27:30.425016 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":102,"threads_started":1}
I20260619 08:27:30.425784 12110 heartbeater.cc:347] Connected to a master server at 127.25.127.190:40447
I20260619 08:27:30.425904 12110 heartbeater.cc:464] Registering TS with master...
I20260619 08:27:30.426096 12110 heartbeater.cc:511] Master 127.25.127.190:40447 requested a full tablet report, sending...
I20260619 08:27:30.426469 11783 ts_manager.cc:194] Registered new tserver with Master: e04dc8fa8f234727be8a65ab3ec81801 (127.25.127.130:38391)
I20260619 08:27:30.426898 11783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:40335
WARNING: no policy specified for kudu/127.25.127.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
I20260619 08:27:30.446883 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:40447
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:42941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:30.532946 12115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:30.533223 12115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:30.533270 12115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:30.534492 12115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:30.534569 12115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:30.534614 12115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:27:30.536170 12115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:40447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:30.536453 12115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:30.536697 12115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:30.541760 12120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:30.541779 12121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:30.541950 12115 server_base.cc:1034] running on GCE node
W20260619 08:27:30.541982 12123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:30.542203 12115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:30.542460 12115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:30.543632 12115 hybrid_clock.cc:648] HybridClock initialized: now 1781857650543628 us; error 37 us; skew 500 ppm
Jun 19 08:27:30 dist-test-slave-5rhg krb5kdc[11738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857650, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:30.546166 12115 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:27:30.546619 12115 webserver.cc:460] Webserver started at http://127.25.127.131:41185/ using document root <none> and password file <none>
I20260619 08:27:30.546871 12115 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:30.546942 12115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:30.547094 12115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:30.548348 12115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "4244825c59854b56a3ec70c70bda321c"
format_stamp: "Formatted at 2026-06-19 08:27:30 on dist-test-slave-5rhg"
server_key: "1c8287d26892f65a3765eee45a5e0ef7"
server_key_iv: "0a4c1d15a5f1e772950380a1f4178a3d"
server_key_version: "encryptionkey@0"
I20260619 08:27:30.548723 12115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "4244825c59854b56a3ec70c70bda321c"
format_stamp: "Formatted at 2026-06-19 08:27:30 on dist-test-slave-5rhg"
server_key: "1c8287d26892f65a3765eee45a5e0ef7"
server_key_iv: "0a4c1d15a5f1e772950380a1f4178a3d"
server_key_version: "encryptionkey@0"
I20260619 08:27:30.550176 12115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:27:30.550891 12130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:30.551062 12115 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:27:30.551169 12115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "4244825c59854b56a3ec70c70bda321c"
format_stamp: "Formatted at 2026-06-19 08:27:30 on dist-test-slave-5rhg"
server_key: "1c8287d26892f65a3765eee45a5e0ef7"
server_key_iv: "0a4c1d15a5f1e772950380a1f4178a3d"
server_key_version: "encryptionkey@0"
I20260619 08:27:30.551263 12115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:30.562652 12115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:30.564731 12115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:30.564899 12115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:30.565136 12115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:30.565461 12115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:30.565514 12115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:30.565557 12115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:30.565585 12115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:30.572065 12115 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:45525
I20260619 08:27:30.572189 12243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:45525 every 8 connection(s)
I20260619 08:27:30.572546 12115 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:27:30.574725 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 12115
I20260619 08:27:30.574818 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:27:30.575014 26110 external_mini_cluster.cc:1428] Setting key 36a8adf842b8dc701d4fc4ce707424dd
Jun 19 08:27:30 dist-test-slave-5rhg krb5kdc[11738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857650, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:27:30.582239 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.573335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:53339 (local address 127.25.127.190:40447)
0619 08:27:30.573401 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:30.573402 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:30.573602 (+ 200us) server_negotiation.cc:407] Connection header received
0619 08:27:30.574094 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:30.574094 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:30.574104 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:30.574164 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:30.575224 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.575863 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.577003 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.577219 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.577302 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.579387 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:30.579389 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:30.579389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:30.579397 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:30.580764 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:30.581191 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:30.581192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:30.581193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:30.581256 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:30.581513 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:30.581514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:30.581515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:30.581648 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:30.581726 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:30.582010 (+ 284us) server_negotiation.cc:299] Negotiation successful
0619 08:27:30.582122 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 08:27:30.582419 12246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.573307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:40447 (local address 127.25.127.131:53339)
0619 08:27:30.573472 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:30.573517 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:30.574013 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:30.574220 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:30.574224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:30.574842 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:30.575113 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:30.575119 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.575994 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.575995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:30.576899 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:30.576902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.577317 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.577318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:30.577452 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.577936 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:27:30.577945 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:30.579293 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:30.580862 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:30.580870 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:30.580872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:30.581109 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:30.581325 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:30.581326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:30.581326 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:30.581445 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:30.581716 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:30.581723 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:30.581874 (+ 151us) client_negotiation.cc:769] Sending connection context
0619 08:27:30.581987 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 08:27:30.582163 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260619 08:27:30.582935 12244 heartbeater.cc:347] Connected to a master server at 127.25.127.190:40447
I20260619 08:27:30.583022 12244 heartbeater.cc:464] Registering TS with master...
I20260619 08:27:30.583230 12244 heartbeater.cc:511] Master 127.25.127.190:40447 requested a full tablet report, sending...
I20260619 08:27:30.583617 11783 ts_manager.cc:194] Registered new tserver with Master: 4244825c59854b56a3ec70c70bda321c (127.25.127.131:45525)
I20260619 08:27:30.583971 11783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:53339
I20260619 08:27:30.587096 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:27:30.594573 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.588470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40896 (local address 127.25.127.190:40447)
0619 08:27:30.588539 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:30.588541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:30.588612 (+ 71us) server_negotiation.cc:407] Connection header received
0619 08:27:30.588673 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:30.588674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:30.588682 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:30.588720 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:30.589313 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.589821 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.590584 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.590767 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.590819 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.592235 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:30.592236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:30.592237 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:30.592242 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:30.593419 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:30.593784 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:30.593784 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:30.593785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:30.593834 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:30.594013 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:30.594014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:30.594014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:30.594150 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:30.594229 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:30.594417 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 08:27:30.594466 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 08:27:30.596448 11783 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40896:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:27:30.598112 11783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:27:30.605110 12259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.600930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50144 (local address 127.25.127.131:45525)
0619 08:27:30.601141 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:30.601144 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:30.601154 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:27:30.601190 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:30.601192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:30.601206 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:30.601275 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:30.601504 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.602584 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.603911 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.604672 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.604754 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.604811 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:30.604967 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 08:27:30.605023 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":96,"threads_started":1}
I20260619 08:27:30.605118 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.600655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:45525 (local address 127.0.0.1:50144)
0619 08:27:30.601061 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:30.601072 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:30.601111 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:30.601351 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:30.601352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:30.601354 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:30.601417 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:30.601419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.602700 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.602701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:30.603819 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:30.603821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.604768 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.604769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:30.604845 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.604859 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 08:27:30.604934 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 08:27:30.604998 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":34,"thread_start_us":60,"threads_started":1}
I20260619 08:27:30.605118 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.600652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:36813 (local address 127.0.0.1:57956)
0619 08:27:30.600828 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:30.600868 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:30.600938 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:30.601206 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:30.601209 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:30.601211 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:30.601288 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:30.601290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.602718 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.602719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:30.604037 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:30.604040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.604881 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.604882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:30.604956 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.604958 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:27:30.605003 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 08:27:30.605035 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":81,"thread_start_us":95,"threads_started":1}
I20260619 08:27:30.605299 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.600841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57956 (local address 127.25.127.129:36813)
0619 08:27:30.601029 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:30.601035 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:30.601046 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:27:30.601090 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:30.601092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:30.601110 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:30.601205 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:30.601432 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.602607 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.604205 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.604698 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.604792 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.604849 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:30.605155 (+ 306us) server_negotiation.cc:299] Negotiation successful
0619 08:27:30.605206 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
I20260619 08:27:30.605912 12178 tablet_service.cc:1467] Processing CreateTablet for tablet adb4a6ca1de44cf68c7aa67dc223bd81 (DEFAULT_TABLE table=test-table [id=cf5e7439ae274642b895f07c674d3edd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:27:30.606233 12260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.601077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44808 (local address 127.25.127.130:38391)
0619 08:27:30.601423 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:30.601428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:30.601554 (+ 126us) server_negotiation.cc:407] Connection header received
0619 08:27:30.601686 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:30.601689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:30.601705 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:30.601799 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:30.601907 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.602625 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.604132 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.604883 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.604970 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.605021 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:30.606094 (+ 1073us) server_negotiation.cc:299] Negotiation successful
0619 08:27:30.606145 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":85,"threads_started":1}
I20260619 08:27:30.606237 12255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.600661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:38391 (local address 127.0.0.1:44808)
0619 08:27:30.601468 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:30.601478 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:30.601607 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:30.601796 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:30.601797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:30.601799 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:30.601850 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:30.601852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.602736 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.602737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:30.604041 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:30.604043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.605058 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.605059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:30.605122 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.605127 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:27:30.606072 (+ 945us) client_negotiation.cc:240] Negotiation successful
0619 08:27:30.606168 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"mutex_wait_us":92,"thread_start_us":95,"threads_started":1}
I20260619 08:27:30.606156 11908 tablet_service.cc:1467] Processing CreateTablet for tablet adb4a6ca1de44cf68c7aa67dc223bd81 (DEFAULT_TABLE table=test-table [id=cf5e7439ae274642b895f07c674d3edd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:27:30.606302 12178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adb4a6ca1de44cf68c7aa67dc223bd81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:30.606444 11908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adb4a6ca1de44cf68c7aa67dc223bd81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:30.607182 12044 tablet_service.cc:1467] Processing CreateTablet for tablet adb4a6ca1de44cf68c7aa67dc223bd81 (DEFAULT_TABLE table=test-table [id=cf5e7439ae274642b895f07c674d3edd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:27:30.607493 12044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adb4a6ca1de44cf68c7aa67dc223bd81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:30.609798 12261 tablet_bootstrap.cc:492] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075: Bootstrap starting.
I20260619 08:27:30.610033 12262 tablet_bootstrap.cc:492] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c: Bootstrap starting.
I20260619 08:27:30.610424 12263 tablet_bootstrap.cc:492] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801: Bootstrap starting.
I20260619 08:27:30.611017 12261 tablet_bootstrap.cc:654] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:30.611030 12263 tablet_bootstrap.cc:654] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:30.611472 12261 log.cc:824] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:30.611519 12263 log.cc:824] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:30.612077 12262 tablet_bootstrap.cc:654] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:30.612362 12261 tablet_bootstrap.cc:492] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075: No bootstrap required, opened a new log
I20260619 08:27:30.612435 12261 ts_tablet_manager.cc:1397] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 08:27:30.612583 12262 log.cc:824] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:30.613229 12262 tablet_bootstrap.cc:492] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c: No bootstrap required, opened a new log
I20260619 08:27:30.613283 12262 ts_tablet_manager.cc:1397] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260619 08:27:30.613976 12261 raft_consensus.cc:348] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4244825c59854b56a3ec70c70bda321c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45525 } } peers { permanent_uuid: "4b86528bbd214985b69d2e0257740075" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36813 } } peers { permanent_uuid: "e04dc8fa8f234727be8a65ab3ec81801" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 38391 } }
I20260619 08:27:30.614148 12261 raft_consensus.cc:374] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:30.614197 12261 raft_consensus.cc:729] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b86528bbd214985b69d2e0257740075, State: Initialized, Role: FOLLOWER
I20260619 08:27:30.614319 12261 consensus_queue.cc:260] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4244825c59854b56a3ec70c70bda321c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45525 } } peers { permanent_uuid: "4b86528bbd214985b69d2e0257740075" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36813 } } peers { permanent_uuid: "e04dc8fa8f234727be8a65ab3ec81801" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 38391 } }
I20260619 08:27:30.614646 12261 ts_tablet_manager.cc:1428] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:27:30.614665 11975 heartbeater.cc:503] Master 127.25.127.190:40447 was elected leader, sending a full tablet report...
I20260619 08:27:30.615124 12263 tablet_bootstrap.cc:492] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801: No bootstrap required, opened a new log
I20260619 08:27:30.615190 12263 ts_tablet_manager.cc:1397] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 08:27:30.615243 12262 raft_consensus.cc:348] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4244825c59854b56a3ec70c70bda321c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45525 } } peers { permanent_uuid: "4b86528bbd214985b69d2e0257740075" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36813 } } peers { permanent_uuid: "e04dc8fa8f234727be8a65ab3ec81801" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 38391 } }
I20260619 08:27:30.615366 12262 raft_consensus.cc:374] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:30.615387 12262 raft_consensus.cc:729] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4244825c59854b56a3ec70c70bda321c, State: Initialized, Role: FOLLOWER
I20260619 08:27:30.615546 12262 consensus_queue.cc:260] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4244825c59854b56a3ec70c70bda321c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45525 } } peers { permanent_uuid: "4b86528bbd214985b69d2e0257740075" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36813 } } peers { permanent_uuid: "e04dc8fa8f234727be8a65ab3ec81801" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 38391 } }
I20260619 08:27:30.615844 12244 heartbeater.cc:503] Master 127.25.127.190:40447 was elected leader, sending a full tablet report...
I20260619 08:27:30.615849 12262 ts_tablet_manager.cc:1428] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260619 08:27:30.617187 12263 raft_consensus.cc:348] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4244825c59854b56a3ec70c70bda321c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45525 } } peers { permanent_uuid: "4b86528bbd214985b69d2e0257740075" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36813 } } peers { permanent_uuid: "e04dc8fa8f234727be8a65ab3ec81801" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 38391 } }
I20260619 08:27:30.617319 12263 raft_consensus.cc:374] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:30.617341 12263 raft_consensus.cc:729] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e04dc8fa8f234727be8a65ab3ec81801, State: Initialized, Role: FOLLOWER
I20260619 08:27:30.617457 12263 consensus_queue.cc:260] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4244825c59854b56a3ec70c70bda321c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45525 } } peers { permanent_uuid: "4b86528bbd214985b69d2e0257740075" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36813 } } peers { permanent_uuid: "e04dc8fa8f234727be8a65ab3ec81801" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 38391 } }
I20260619 08:27:30.617717 12263 ts_tablet_manager.cc:1428] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:27:30.617784 12110 heartbeater.cc:503] Master 127.25.127.190:40447 was elected leader, sending a full tablet report...
I20260619 08:27:30.659543 12267 raft_consensus.cc:482] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:27:30.659677 12267 raft_consensus.cc:504] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4244825c59854b56a3ec70c70bda321c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45525 } } peers { permanent_uuid: "4b86528bbd214985b69d2e0257740075" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36813 } } peers { permanent_uuid: "e04dc8fa8f234727be8a65ab3ec81801" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 38391 } }
I20260619 08:27:30.660149 12267 leader_election.cc:288] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4244825c59854b56a3ec70c70bda321c (127.25.127.131:45525), e04dc8fa8f234727be8a65ab3ec81801 (127.25.127.130:38391)
I20260619 08:27:30.663112 12260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.660442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:43115 (local address 127.25.127.130:38391)
0619 08:27:30.660489 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:30.660491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:30.660501 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:27:30.660582 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:30.660584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:30.660609 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:30.660687 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:30.660856 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.661345 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.662329 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.662733 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.662826 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.662835 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:30.662991 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 08:27:30.663034 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:27:30.663123 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.660345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:38391 (local address 127.25.127.129:43115)
0619 08:27:30.660426 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:30.660439 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:30.660500 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:30.660694 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:30.660695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:30.660697 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:30.660779 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:30.660781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.661433 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.661434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:30.662224 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:30.662226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.662823 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.662824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:30.662913 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.662916 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:27:30.662985 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 08:27:30.663038 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 08:27:30.663156 12259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.660470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:44179 (local address 127.25.127.131:45525)
0619 08:27:30.660519 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:30.660521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:30.660650 (+ 129us) server_negotiation.cc:407] Connection header received
0619 08:27:30.660682 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:30.660683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:30.660692 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:30.660766 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:30.660927 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.661398 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.662383 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.662785 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.662874 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.662883 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:30.663042 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 08:27:30.663081 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:27:30.663210 12270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.660366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:45525 (local address 127.25.127.129:44179)
0619 08:27:30.660558 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:30.660566 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:30.660623 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:30.660771 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:30.660772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:30.660773 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:30.660854 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:30.660855 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.661484 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.661485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:30.662277 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:30.662280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.662873 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.662874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:30.662952 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.662956 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:27:30.663036 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 08:27:30.663080 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":118,"threads_started":1}
I20260619 08:27:30.663622 12198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adb4a6ca1de44cf68c7aa67dc223bd81" candidate_uuid: "4b86528bbd214985b69d2e0257740075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4244825c59854b56a3ec70c70bda321c" is_pre_election: true
I20260619 08:27:30.663669 12064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adb4a6ca1de44cf68c7aa67dc223bd81" candidate_uuid: "4b86528bbd214985b69d2e0257740075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e04dc8fa8f234727be8a65ab3ec81801" is_pre_election: true
I20260619 08:27:30.663800 12064 raft_consensus.cc:2454] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b86528bbd214985b69d2e0257740075 in term 0.
I20260619 08:27:30.663800 12198 raft_consensus.cc:2454] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b86528bbd214985b69d2e0257740075 in term 0.
I20260619 08:27:30.664064 11862 leader_election.cc:302] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4244825c59854b56a3ec70c70bda321c, 4b86528bbd214985b69d2e0257740075; no voters:
I20260619 08:27:30.664242 12267 raft_consensus.cc:2781] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:27:30.664305 12267 raft_consensus.cc:482] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:27:30.664321 12267 raft_consensus.cc:3037] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:30.665081 12267 raft_consensus.cc:504] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4244825c59854b56a3ec70c70bda321c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45525 } } peers { permanent_uuid: "4b86528bbd214985b69d2e0257740075" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36813 } } peers { permanent_uuid: "e04dc8fa8f234727be8a65ab3ec81801" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 38391 } }
I20260619 08:27:30.665243 12267 leader_election.cc:288] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [CANDIDATE]: Term 1 election: Requested vote from peers 4244825c59854b56a3ec70c70bda321c (127.25.127.131:45525), e04dc8fa8f234727be8a65ab3ec81801 (127.25.127.130:38391)
I20260619 08:27:30.665416 12064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adb4a6ca1de44cf68c7aa67dc223bd81" candidate_uuid: "4b86528bbd214985b69d2e0257740075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e04dc8fa8f234727be8a65ab3ec81801"
I20260619 08:27:30.665465 12198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adb4a6ca1de44cf68c7aa67dc223bd81" candidate_uuid: "4b86528bbd214985b69d2e0257740075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4244825c59854b56a3ec70c70bda321c"
I20260619 08:27:30.665536 12064 raft_consensus.cc:3037] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:30.665563 12198 raft_consensus.cc:3037] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c [term 0 FOLLOWER]: Advancing to term 1
W20260619 08:27:30.665664 12111 tablet.cc:2369] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:27:30.666389 12064 raft_consensus.cc:2454] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b86528bbd214985b69d2e0257740075 in term 1.
I20260619 08:27:30.666394 12198 raft_consensus.cc:2454] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b86528bbd214985b69d2e0257740075 in term 1.
I20260619 08:27:30.666589 11863 leader_election.cc:302] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b86528bbd214985b69d2e0257740075, e04dc8fa8f234727be8a65ab3ec81801; no voters:
I20260619 08:27:30.666703 12267 raft_consensus.cc:2781] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:30.666935 12267 raft_consensus.cc:686] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [term 1 LEADER]: Becoming Leader. State: Replica: 4b86528bbd214985b69d2e0257740075, State: Running, Role: LEADER
I20260619 08:27:30.667055 12267 consensus_queue.cc:237] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4244825c59854b56a3ec70c70bda321c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45525 } } peers { permanent_uuid: "4b86528bbd214985b69d2e0257740075" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36813 } } peers { permanent_uuid: "e04dc8fa8f234727be8a65ab3ec81801" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 38391 } }
I20260619 08:27:30.668008 11783 catalog_manager.cc:5495] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b86528bbd214985b69d2e0257740075 (127.25.127.129). New cstate: current_term: 1 leader_uuid: "4b86528bbd214985b69d2e0257740075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4244825c59854b56a3ec70c70bda321c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b86528bbd214985b69d2e0257740075" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e04dc8fa8f234727be8a65ab3ec81801" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 38391 } health_report { overall_health: UNKNOWN } } }
I20260619 08:27:30.678498 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.676019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57966 (local address 127.25.127.129:36813)
0619 08:27:30.676070 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:30.676071 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:30.676078 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:27:30.676110 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:30.676111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:30.676127 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:30.676236 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:27:30.676384 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.676879 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.677741 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.677936 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.678028 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.678136 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:27:30.678229 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:27:30.678301 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:30.678346 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 08:27:30.678356 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 08:27:30.680366 12198 raft_consensus.cc:1261] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c [term 1 FOLLOWER]: Refusing update from remote peer 4b86528bbd214985b69d2e0257740075: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:27:30.680382 12064 raft_consensus.cc:1261] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801 [term 1 FOLLOWER]: Refusing update from remote peer 4b86528bbd214985b69d2e0257740075: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:27:30.680779 12271 consensus_queue.cc:1035] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4244825c59854b56a3ec70c70bda321c" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 45525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:27:30.680960 12267 consensus_queue.cc:1035] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e04dc8fa8f234727be8a65ab3ec81801" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 38391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:27:30.683506 12275 mvcc.cc:205] Tried to move back new op lower bound from 7298488937184178176 to 7298488937133092864. Current Snapshot: MvccSnapshot[applied={T|T < 7298488937184178176}]
I20260619 08:27:30.685976 11782 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:27:30.686086 11782 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:27:30.686939 11782 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca62df64bcb844acbeb5e7d6d3957a1e: Sending DeleteTablet for 3 replicas of tablet adb4a6ca1de44cf68c7aa67dc223bd81
I20260619 08:27:30.687335 11908 tablet_service.cc:1514] Processing DeleteTablet for tablet adb4a6ca1de44cf68c7aa67dc223bd81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:27:30 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:57956
I20260619 08:27:30.687332 12178 tablet_service.cc:1514] Processing DeleteTablet for tablet adb4a6ca1de44cf68c7aa67dc223bd81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:27:30 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:50144
I20260619 08:27:30.687687 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 11845
I20260619 08:27:30.687736 12284 tablet_replica.cc:330] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4b86528bbd214985b69d2e0257740075: stopping tablet replica
I20260619 08:27:30.688361 12044 tablet_service.cc:1514] Processing DeleteTablet for tablet adb4a6ca1de44cf68c7aa67dc223bd81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:27:30 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:44808
I20260619 08:27:30.688627 12285 tablet_replica.cc:330] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801: stopping tablet replica
I20260619 08:27:30.688761 12285 raft_consensus.cc:2229] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:27:30.688822 12283 tablet_replica.cc:330] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c: stopping tablet replica
I20260619 08:27:30.688882 12285 raft_consensus.cc:2258] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:27:30.688916 12283 raft_consensus.cc:2229] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:27:30.689157 12283 raft_consensus.cc:2258] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:27:30.689335 12285 ts_tablet_manager.cc:1905] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:27:30.689620 12283 ts_tablet_manager.cc:1905] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:27:30.691257 12285 ts_tablet_manager.cc:1918] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:27:30.691366 12285 log.cc:1196] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/adb4a6ca1de44cf68c7aa67dc223bd81
I20260619 08:27:30.691365 12283 ts_tablet_manager.cc:1918] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:27:30.691424 12283 log.cc:1196] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/adb4a6ca1de44cf68c7aa67dc223bd81
I20260619 08:27:30.691681 12285 ts_tablet_manager.cc:1939] T adb4a6ca1de44cf68c7aa67dc223bd81 P e04dc8fa8f234727be8a65ab3ec81801: Deleting consensus metadata
I20260619 08:27:30.691694 12283 ts_tablet_manager.cc:1939] T adb4a6ca1de44cf68c7aa67dc223bd81 P 4244825c59854b56a3ec70c70bda321c: Deleting consensus metadata
I20260619 08:27:30.692096 11768 catalog_manager.cc:4841] TS 4244825c59854b56a3ec70c70bda321c (127.25.127.131:45525): tablet adb4a6ca1de44cf68c7aa67dc223bd81 (table test-table [id=cf5e7439ae274642b895f07c674d3edd]) successfully deleted
I20260619 08:27:30.692188 11769 catalog_manager.cc:4841] TS e04dc8fa8f234727be8a65ab3ec81801 (127.25.127.130:38391): tablet adb4a6ca1de44cf68c7aa67dc223bd81 (table test-table [id=cf5e7439ae274642b895f07c674d3edd]) successfully deleted
W20260619 08:27:30.694904 11767 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.127.129:36813 (error 108)
I20260619 08:27:30.695145 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 11980
I20260619 08:27:30.695988 12255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.695827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:36813 (local address 127.0.0.1:57974)
0619 08:27:30.695901 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:30.695945 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.129:36813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 08:27:30.696142 11767 catalog_manager.cc:4584] TS 4b86528bbd214985b69d2e0257740075 (127.25.127.129:36813): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet adb4a6ca1de44cf68c7aa67dc223bd81: Network error: Client connection negotiation failed: client connection to 127.25.127.129:36813: connect: Connection refused (error 111)
I20260619 08:27:30.699905 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 12115
I20260619 08:27:30.704900 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 11751
2026-06-19T08:27:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1949 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T08:27:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:30Z Disabled control of system clock
I20260619 08:27:30.733825 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:42815
--webserver_interface=127.25.127.190
--webserver_port=0
--builtin_ntp_servers=127.25.127.148:41575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:42815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:27:30.815016 12290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:30.815268 12290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:30.815290 12290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:30.816464 12290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:30.816502 12290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:30.816514 12290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:30.816527 12290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:30.816540 12290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:30.817836 12290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:41575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:42815
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:42815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:30.818042 12290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:30.818302 12290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:30.823105 12295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:30.823136 12296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:30.823366 12290 server_base.cc:1034] running on GCE node
W20260619 08:27:30.823503 12298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:30.823791 12290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:30.824100 12290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:30.825300 12290 hybrid_clock.cc:648] HybridClock initialized: now 1781857650825276 us; error 39 us; skew 500 ppm
I20260619 08:27:30.826524 12290 webserver.cc:460] Webserver started at http://127.25.127.190:45369/ using document root <none> and password file <none>
I20260619 08:27:30.826808 12290 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:30.826880 12290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:30.827036 12290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:30.828228 12290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "b6e6138e4a2141ec86e1d5ec585515a3"
format_stamp: "Formatted at 2026-06-19 08:27:30 on dist-test-slave-5rhg"
server_key: "22e0cb792b09e3ba922fb97fb956510c"
server_key_iv: "7388622ed2c5133803152ad9c2c710ee"
server_key_version: "encryptionkey@0"
I20260619 08:27:30.828596 12290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "b6e6138e4a2141ec86e1d5ec585515a3"
format_stamp: "Formatted at 2026-06-19 08:27:30 on dist-test-slave-5rhg"
server_key: "22e0cb792b09e3ba922fb97fb956510c"
server_key_iv: "7388622ed2c5133803152ad9c2c710ee"
server_key_version: "encryptionkey@0"
I20260619 08:27:30.829887 12290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:27:30.830646 12304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:30.830843 12290 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:30.830942 12290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "b6e6138e4a2141ec86e1d5ec585515a3"
format_stamp: "Formatted at 2026-06-19 08:27:30 on dist-test-slave-5rhg"
server_key: "22e0cb792b09e3ba922fb97fb956510c"
server_key_iv: "7388622ed2c5133803152ad9c2c710ee"
server_key_version: "encryptionkey@0"
I20260619 08:27:30.831025 12290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:30.844667 12290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:30.844986 12290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:30.845139 12290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:30.849220 12290 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:42815
I20260619 08:27:30.849257 12356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:42815 every 8 connection(s)
I20260619 08:27:30.849603 12290 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:30.850620 12357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:30.853057 12357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3: Bootstrap starting.
I20260619 08:27:30.853667 12357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:30.853984 12357 log.cc:824] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:30.854668 12357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3: No bootstrap required, opened a new log
I20260619 08:27:30.856374 12357 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6e6138e4a2141ec86e1d5ec585515a3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 42815 } }
I20260619 08:27:30.856513 12357 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:30.856536 12357 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6e6138e4a2141ec86e1d5ec585515a3, State: Initialized, Role: FOLLOWER
I20260619 08:27:30.856710 12357 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6e6138e4a2141ec86e1d5ec585515a3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 42815 } }
I20260619 08:27:30.856778 12357 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:30.856812 12357 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:30.856853 12357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:30.857415 12357 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6e6138e4a2141ec86e1d5ec585515a3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 42815 } }
I20260619 08:27:30.857528 12357 leader_election.cc:302] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6e6138e4a2141ec86e1d5ec585515a3; no voters:
I20260619 08:27:30.857702 12357 leader_election.cc:288] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:30.857730 12360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:30.857914 12360 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3 [term 1 LEADER]: Becoming Leader. State: Replica: b6e6138e4a2141ec86e1d5ec585515a3, State: Running, Role: LEADER
I20260619 08:27:30.858179 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 12290
I20260619 08:27:30.858309 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:30.858522 26110 external_mini_cluster.cc:1428] Setting key 08cae1530123c990b8059355937c7b26
I20260619 08:27:30.858016 12357 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:30.858806 12360 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6e6138e4a2141ec86e1d5ec585515a3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 42815 } }
I20260619 08:27:30.859846 12362 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6e6138e4a2141ec86e1d5ec585515a3. Latest consensus state: current_term: 1 leader_uuid: "b6e6138e4a2141ec86e1d5ec585515a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6e6138e4a2141ec86e1d5ec585515a3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 42815 } } }
I20260619 08:27:30.859943 12362 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:30.860189 12372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:30.860318 12361 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6e6138e4a2141ec86e1d5ec585515a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6e6138e4a2141ec86e1d5ec585515a3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 42815 } } }
I20260619 08:27:30.860474 12361 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:30.860719 12372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:30.862224 12372 catalog_manager.cc:1347] Generated new cluster ID: 6420d6baa6da4c68bc4ba6d7a71a59c4
I20260619 08:27:30.862286 12372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:30.865124 12366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:30.859157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48644 (local address 127.25.127.190:42815)
0619 08:27:30.859575 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:30.859585 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:30.859789 (+ 204us) server_negotiation.cc:407] Connection header received
0619 08:27:30.860230 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:30.860239 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:30.860314 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:30.860725 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:30.860824 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.862235 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.863040 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:30.863663 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:30.863778 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:30.864458 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:30.864462 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:30.864462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:30.864673 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:30.864714 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:30.864727 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:30.864788 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:30.864845 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 08:27:30.864920 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":61,"threads_started":1}
I20260619 08:27:30.873518 12372 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:30.874029 12372 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:30.883427 12372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3: Generated new TSK 0
I20260619 08:27:30.883669 12372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:30.892006 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:42815
--builtin_ntp_servers=127.25.127.148:41575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:27:30.974843 12381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:30.975111 12381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:30.975148 12381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:30.976444 12381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:30.976508 12381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:30.976549 12381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:27:30.977977 12381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:41575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:42815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:30.978253 12381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:30.978559 12381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:30.983770 12389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:30.983814 12386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:30.983922 12387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:30.984225 12381 server_base.cc:1034] running on GCE node
I20260619 08:27:30.984450 12381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:30.984751 12381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:30.985914 12381 hybrid_clock.cc:648] HybridClock initialized: now 1781857650985896 us; error 34 us; skew 500 ppm
I20260619 08:27:30.987303 12381 webserver.cc:460] Webserver started at http://127.25.127.129:33259/ using document root <none> and password file <none>
I20260619 08:27:30.987560 12381 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:30.987653 12381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:30.987805 12381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:30.988971 12381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "73af5c5fdd924807ba6c8e614c0d3b15"
format_stamp: "Formatted at 2026-06-19 08:27:30 on dist-test-slave-5rhg"
server_key: "1285a5203a471a94e8992eca36bc7994"
server_key_iv: "afe4793b4d09c423ae9fdc7dcab3a7c9"
server_key_version: "encryptionkey@0"
I20260619 08:27:30.989346 12381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "73af5c5fdd924807ba6c8e614c0d3b15"
format_stamp: "Formatted at 2026-06-19 08:27:30 on dist-test-slave-5rhg"
server_key: "1285a5203a471a94e8992eca36bc7994"
server_key_iv: "afe4793b4d09c423ae9fdc7dcab3a7c9"
server_key_version: "encryptionkey@0"
I20260619 08:27:30.990679 12381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:27:30.991437 12395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:30.991590 12381 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:27:30.991696 12381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "73af5c5fdd924807ba6c8e614c0d3b15"
format_stamp: "Formatted at 2026-06-19 08:27:30 on dist-test-slave-5rhg"
server_key: "1285a5203a471a94e8992eca36bc7994"
server_key_iv: "afe4793b4d09c423ae9fdc7dcab3a7c9"
server_key_version: "encryptionkey@0"
I20260619 08:27:30.991772 12381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:31.000924 12381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:31.001217 12381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:31.001360 12381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:31.001576 12381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:31.001896 12381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:31.001928 12381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:31.001951 12381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:31.001994 12381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:31.007910 12381 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:42029
I20260619 08:27:31.007951 12508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:42029 every 8 connection(s)
I20260619 08:27:31.008332 12381 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:27:31.013499 12366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.009053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:41349 (local address 127.25.127.190:42815)
0619 08:27:31.009127 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:31.009129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:31.009359 (+ 230us) server_negotiation.cc:407] Connection header received
0619 08:27:31.009836 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:31.009837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:31.009850 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:31.009902 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:31.010529 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.011020 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.012006 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.012322 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.012389 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:31.013251 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:31.013251 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:31.013252 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:31.013256 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:31.013270 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:31.013273 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:31.013322 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:31.013390 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 08:27:31.013427 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:27:31.013710 12511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.009053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:42815 (local address 127.25.127.129:41349)
0619 08:27:31.009227 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:31.009260 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:31.009749 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:31.009956 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:31.009959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:31.009967 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:31.010456 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:31.010461 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.011094 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.011095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:31.011897 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:31.011899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.012406 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.012407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:31.012520 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:31.013108 (+ 588us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:27:31.013109 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:31.013140 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:27:31.013164 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:31.013322 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:31.013324 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:31.013327 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:27:31.013375 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 08:27:31.013460 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260619 08:27:31.014271 12509 heartbeater.cc:347] Connected to a master server at 127.25.127.190:42815
I20260619 08:27:31.014370 12509 heartbeater.cc:464] Registering TS with master...
I20260619 08:27:31.014530 12509 heartbeater.cc:511] Master 127.25.127.190:42815 requested a full tablet report, sending...
I20260619 08:27:31.015039 12321 ts_manager.cc:194] Registered new tserver with Master: 73af5c5fdd924807ba6c8e614c0d3b15 (127.25.127.129:42029)
I20260619 08:27:31.015669 12321 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.127.129:41349
I20260619 08:27:31.017297 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 12381
I20260619 08:27:31.017386 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:27:31.017549 26110 external_mini_cluster.cc:1428] Setting key 38af8f0a106d30bec2b304e01c9653be
I20260619 08:27:31.018448 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:42815
--builtin_ntp_servers=127.25.127.148:41575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:27:31.101491 12512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:31.101716 12512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:31.101742 12512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:31.102921 12512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:31.102963 12512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:31.102991 12512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:27:31.104455 12512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:41575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:42815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:31.104715 12512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:31.105031 12512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:31.110186 12518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:31.110196 12517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:31.110196 12520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:31.110524 12512 server_base.cc:1034] running on GCE node
I20260619 08:27:31.110767 12512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:31.111023 12512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:31.112232 12512 hybrid_clock.cc:648] HybridClock initialized: now 1781857651112211 us; error 29 us; skew 500 ppm
I20260619 08:27:31.117396 12512 webserver.cc:460] Webserver started at http://127.25.127.130:37545/ using document root <none> and password file <none>
I20260619 08:27:31.117688 12512 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:31.117765 12512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:31.117923 12512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:31.119067 12512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "559aaf76a95d4e80ab3853d7d20b712a"
format_stamp: "Formatted at 2026-06-19 08:27:31 on dist-test-slave-5rhg"
server_key: "507dfca6c3a7650aaf9e398813bc0e38"
server_key_iv: "cfd7040c7f0cb3a8954e1f2d803bec58"
server_key_version: "encryptionkey@0"
I20260619 08:27:31.119437 12512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "559aaf76a95d4e80ab3853d7d20b712a"
format_stamp: "Formatted at 2026-06-19 08:27:31 on dist-test-slave-5rhg"
server_key: "507dfca6c3a7650aaf9e398813bc0e38"
server_key_iv: "cfd7040c7f0cb3a8954e1f2d803bec58"
server_key_version: "encryptionkey@0"
I20260619 08:27:31.120988 12512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:27:31.121840 12526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:31.122035 12512 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:31.122159 12512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "559aaf76a95d4e80ab3853d7d20b712a"
format_stamp: "Formatted at 2026-06-19 08:27:31 on dist-test-slave-5rhg"
server_key: "507dfca6c3a7650aaf9e398813bc0e38"
server_key_iv: "cfd7040c7f0cb3a8954e1f2d803bec58"
server_key_version: "encryptionkey@0"
I20260619 08:27:31.122242 12512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:31.141865 12512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:31.142225 12512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:31.142385 12512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:31.142629 12512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:31.142987 12512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:31.143046 12512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:31.143088 12512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:31.143116 12512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:31.149358 12512 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:45619
I20260619 08:27:31.149408 12639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:45619 every 8 connection(s)
I20260619 08:27:31.149787 12512 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:27:31.153434 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 12512
I20260619 08:27:31.153563 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:27:31.153770 26110 external_mini_cluster.cc:1428] Setting key 7a57d68ce98d4f2085b413a239962412
I20260619 08:27:31.154903 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:42815
--builtin_ntp_servers=127.25.127.148:41575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 08:27:31.155669 12366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.150650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:36475 (local address 127.25.127.190:42815)
0619 08:27:31.150718 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:31.150720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:31.150914 (+ 194us) server_negotiation.cc:407] Connection header received
0619 08:27:31.151419 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:31.151421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:31.151432 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:31.151517 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:31.152333 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.152859 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.153937 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.154208 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.154269 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:31.155333 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:31.155334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:31.155335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:31.155340 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:31.155359 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:31.155364 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:31.155427 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:31.155508 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 08:27:31.155540 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:27:31.155792 12642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.150570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:42815 (local address 127.25.127.130:36475)
0619 08:27:31.150779 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:31.150822 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:31.151324 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:31.151607 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:31.151611 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:31.151627 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:31.152230 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:31.152237 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.152949 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.152949 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:31.153854 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:31.153857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.154302 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.154303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:31.154473 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:31.155180 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:27:31.155180 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:31.155212 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:27:31.155238 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:31.155434 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:31.155438 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:31.155441 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:27:31.155495 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:27:31.155583 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
I20260619 08:27:31.156356 12640 heartbeater.cc:347] Connected to a master server at 127.25.127.190:42815
I20260619 08:27:31.156481 12640 heartbeater.cc:464] Registering TS with master...
I20260619 08:27:31.156708 12640 heartbeater.cc:511] Master 127.25.127.190:42815 requested a full tablet report, sending...
I20260619 08:27:31.157112 12321 ts_manager.cc:194] Registered new tserver with Master: 559aaf76a95d4e80ab3853d7d20b712a (127.25.127.130:45619)
I20260619 08:27:31.157615 12321 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.127.130:36475
W20260619 08:27:31.237632 12643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:31.237846 12643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:31.237867 12643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:31.238956 12643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:31.238991 12643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:31.239015 12643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:27:31.240391 12643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:41575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:42815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:31.240618 12643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:31.240911 12643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:31.245899 12649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:31.246049 12643 server_base.cc:1034] running on GCE node
W20260619 08:27:31.245995 12648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:31.245995 12651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:31.246403 12643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:31.246658 12643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:31.247828 12643 hybrid_clock.cc:648] HybridClock initialized: now 1781857651247829 us; error 41 us; skew 500 ppm
I20260619 08:27:31.249135 12643 webserver.cc:460] Webserver started at http://127.25.127.131:40175/ using document root <none> and password file <none>
I20260619 08:27:31.249379 12643 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:31.249449 12643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:31.249601 12643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:31.250736 12643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "98aa448ca86c44b7ab45da808384a3bd"
format_stamp: "Formatted at 2026-06-19 08:27:31 on dist-test-slave-5rhg"
server_key: "189e0a40d25878e056fd787efd807ee6"
server_key_iv: "297e98743008e188ae1f24785b1df711"
server_key_version: "encryptionkey@0"
I20260619 08:27:31.251098 12643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "98aa448ca86c44b7ab45da808384a3bd"
format_stamp: "Formatted at 2026-06-19 08:27:31 on dist-test-slave-5rhg"
server_key: "189e0a40d25878e056fd787efd807ee6"
server_key_iv: "297e98743008e188ae1f24785b1df711"
server_key_version: "encryptionkey@0"
I20260619 08:27:31.252480 12643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:27:31.253240 12657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:31.253415 12643 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:27:31.253521 12643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "98aa448ca86c44b7ab45da808384a3bd"
format_stamp: "Formatted at 2026-06-19 08:27:31 on dist-test-slave-5rhg"
server_key: "189e0a40d25878e056fd787efd807ee6"
server_key_iv: "297e98743008e188ae1f24785b1df711"
server_key_version: "encryptionkey@0"
I20260619 08:27:31.253598 12643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:31.264029 12643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:31.264365 12643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:31.264524 12643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:31.264744 12643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:31.265065 12643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:31.265132 12643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:31.265177 12643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:31.265204 12643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:31.271189 12643 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:43239
I20260619 08:27:31.271230 12770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:43239 every 8 connection(s)
I20260619 08:27:31.271576 12643 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:27:31.277020 12366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.272346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:48293 (local address 127.25.127.190:42815)
0619 08:27:31.272390 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:31.272392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:31.272627 (+ 235us) server_negotiation.cc:407] Connection header received
0619 08:27:31.273188 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:31.273189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:31.273203 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:31.273270 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:31.273948 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.274463 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.275462 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.275686 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.275775 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:31.276717 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:31.276718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:31.276719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:31.276724 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:31.276741 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:31.276745 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:31.276800 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:31.276900 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 08:27:31.276938 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:27:31.277258 12773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.272328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:42815 (local address 127.25.127.131:48293)
0619 08:27:31.272493 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:31.272527 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:31.273051 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:31.273349 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:31.273352 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:31.273360 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:31.273850 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:31.273855 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.274562 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.274563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:31.275364 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:31.275366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.275789 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.275790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:31.275925 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:31.276548 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:27:31.276549 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:31.276578 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:27:31.276603 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:31.276805 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:31.276811 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:31.276813 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:27:31.276904 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 08:27:31.277013 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":66,"threads_started":1}
I20260619 08:27:31.277797 12771 heartbeater.cc:347] Connected to a master server at 127.25.127.190:42815
I20260619 08:27:31.277905 12771 heartbeater.cc:464] Registering TS with master...
I20260619 08:27:31.278102 12771 heartbeater.cc:511] Master 127.25.127.190:42815 requested a full tablet report, sending...
I20260619 08:27:31.278499 12321 ts_manager.cc:194] Registered new tserver with Master: 98aa448ca86c44b7ab45da808384a3bd (127.25.127.131:43239)
I20260619 08:27:31.278846 12321 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.127.131:48293
I20260619 08:27:31.280756 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 12643
I20260619 08:27:31.280845 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:27:31.280987 26110 external_mini_cluster.cc:1428] Setting key 32b4206af87252ca7cd75254d7aa54cc
I20260619 08:27:31.281759 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:27:31.286176 12366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.283071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48650 (local address 127.25.127.190:42815)
0619 08:27:31.283156 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:31.283158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:31.283249 (+ 91us) server_negotiation.cc:407] Connection header received
0619 08:27:31.283328 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:31.283329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:31.283341 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:31.283405 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:31.283581 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.284061 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.284867 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.285071 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.285165 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:31.285843 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:31.285845 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:31.285845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:31.285849 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:31.285864 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:31.285869 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:31.285970 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:31.286070 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 08:27:31.286082 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 08:27:31.287959 12321 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:27:31.289604 12321 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:27:31.296140 12782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.291982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39474 (local address 127.25.127.129:42029)
0619 08:27:31.292273 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:31.292277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:31.292290 (+ 13us) server_negotiation.cc:407] Connection header received
0619 08:27:31.292415 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:31.292418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:31.292431 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:31.292517 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:31.292773 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.293574 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.295113 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.295735 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.295822 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:31.295919 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:31.296007 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 08:27:31.296047 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":87,"threads_started":1}
I20260619 08:27:31.296130 12780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.291881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:42029 (local address 127.0.0.1:39474)
0619 08:27:31.292213 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:31.292247 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:31.292312 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:31.292604 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:31.292607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:31.292609 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:31.292683 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:31.292686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.293666 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.293667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:31.295023 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:31.295025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.295830 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.295831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:31.295907 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:31.295920 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 08:27:31.295993 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 08:27:31.296046 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":39,"thread_start_us":90,"threads_started":1}
I20260619 08:27:31.296468 12779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.291875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:45619 (local address 127.0.0.1:46372)
0619 08:27:31.292198 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:31.292241 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:31.292313 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:31.292639 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:31.292640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:31.292642 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:31.292694 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:31.292697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.294016 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.294017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:31.295153 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:31.295155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.296202 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.296203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:31.296271 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:31.296276 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:27:31.296334 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 08:27:31.296380 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":16,"thread_start_us":85,"threads_started":1}
I20260619 08:27:31.296499 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.291975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46372 (local address 127.25.127.130:45619)
0619 08:27:31.292169 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:31.292174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:31.292401 (+ 227us) server_negotiation.cc:407] Connection header received
0619 08:27:31.292452 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:31.292455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:31.292473 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:31.292544 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:31.292777 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.293925 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.295234 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.295966 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.296040 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:31.296165 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:31.296372 (+ 207us) server_negotiation.cc:299] Negotiation successful
0619 08:27:31.296419 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
I20260619 08:27:31.296797 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.292348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41566 (local address 127.25.127.131:43239)
0619 08:27:31.292562 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:31.292568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:31.292784 (+ 216us) server_negotiation.cc:407] Connection header received
0619 08:27:31.292831 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:31.292833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:31.292847 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:31.292921 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:31.293041 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.294115 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.295379 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.296071 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.296131 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:31.296509 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:31.296537 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 08:27:31.296719 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":113,"threads_started":1}
I20260619 08:27:31.296478 12783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.292133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:43239 (local address 127.0.0.1:41566)
0619 08:27:31.292660 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:31.292670 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:31.292722 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:31.292937 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:31.292937 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:31.292940 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:31.292974 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:31.292976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.294228 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.294229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:31.295297 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:31.295300 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.296285 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.296286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:31.296346 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:31.296349 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:27:31.296387 (+ 38us) client_negotiation.cc:240] Negotiation successful
0619 08:27:31.296417 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":75,"threads_started":1}
I20260619 08:27:31.297273 12574 tablet_service.cc:1467] Processing CreateTablet for tablet f0d32be4adff4d7eb25e47533f16fbdb (DEFAULT_TABLE table=test-table [id=16b88ccdc2f246cdb9cb25b2308d9386]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:27:31.297457 12705 tablet_service.cc:1467] Processing CreateTablet for tablet f0d32be4adff4d7eb25e47533f16fbdb (DEFAULT_TABLE table=test-table [id=16b88ccdc2f246cdb9cb25b2308d9386]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:27:31.297446 12443 tablet_service.cc:1467] Processing CreateTablet for tablet f0d32be4adff4d7eb25e47533f16fbdb (DEFAULT_TABLE table=test-table [id=16b88ccdc2f246cdb9cb25b2308d9386]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:27:31.297667 12574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0d32be4adff4d7eb25e47533f16fbdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:31.297760 12705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0d32be4adff4d7eb25e47533f16fbdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:31.297842 12443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0d32be4adff4d7eb25e47533f16fbdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:31.300485 12785 tablet_bootstrap.cc:492] T f0d32be4adff4d7eb25e47533f16fbdb P 73af5c5fdd924807ba6c8e614c0d3b15: Bootstrap starting.
I20260619 08:27:31.301082 12786 tablet_bootstrap.cc:492] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd: Bootstrap starting.
I20260619 08:27:31.301316 12787 tablet_bootstrap.cc:492] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a: Bootstrap starting.
I20260619 08:27:31.301371 12785 tablet_bootstrap.cc:654] T f0d32be4adff4d7eb25e47533f16fbdb P 73af5c5fdd924807ba6c8e614c0d3b15: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:31.301712 12785 log.cc:824] T f0d32be4adff4d7eb25e47533f16fbdb P 73af5c5fdd924807ba6c8e614c0d3b15: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:31.301985 12787 tablet_bootstrap.cc:654] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:31.302330 12787 log.cc:824] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:31.302515 12785 tablet_bootstrap.cc:492] T f0d32be4adff4d7eb25e47533f16fbdb P 73af5c5fdd924807ba6c8e614c0d3b15: No bootstrap required, opened a new log
I20260619 08:27:31.302595 12785 ts_tablet_manager.cc:1397] T f0d32be4adff4d7eb25e47533f16fbdb P 73af5c5fdd924807ba6c8e614c0d3b15: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:27:31.303056 12787 tablet_bootstrap.cc:492] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a: No bootstrap required, opened a new log
I20260619 08:27:31.303151 12787 ts_tablet_manager.cc:1397] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:27:31.303188 12786 tablet_bootstrap.cc:654] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:31.303520 12786 log.cc:824] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:31.304945 12785 raft_consensus.cc:348] T f0d32be4adff4d7eb25e47533f16fbdb P 73af5c5fdd924807ba6c8e614c0d3b15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559aaf76a95d4e80ab3853d7d20b712a" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45619 } } peers { permanent_uuid: "73af5c5fdd924807ba6c8e614c0d3b15" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 42029 } } peers { permanent_uuid: "98aa448ca86c44b7ab45da808384a3bd" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 43239 } }
I20260619 08:27:31.305078 12785 raft_consensus.cc:374] T f0d32be4adff4d7eb25e47533f16fbdb P 73af5c5fdd924807ba6c8e614c0d3b15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:31.305101 12785 raft_consensus.cc:729] T f0d32be4adff4d7eb25e47533f16fbdb P 73af5c5fdd924807ba6c8e614c0d3b15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73af5c5fdd924807ba6c8e614c0d3b15, State: Initialized, Role: FOLLOWER
I20260619 08:27:31.305246 12785 consensus_queue.cc:260] T f0d32be4adff4d7eb25e47533f16fbdb P 73af5c5fdd924807ba6c8e614c0d3b15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559aaf76a95d4e80ab3853d7d20b712a" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45619 } } peers { permanent_uuid: "73af5c5fdd924807ba6c8e614c0d3b15" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 42029 } } peers { permanent_uuid: "98aa448ca86c44b7ab45da808384a3bd" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 43239 } }
I20260619 08:27:31.305354 12787 raft_consensus.cc:348] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559aaf76a95d4e80ab3853d7d20b712a" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45619 } } peers { permanent_uuid: "73af5c5fdd924807ba6c8e614c0d3b15" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 42029 } } peers { permanent_uuid: "98aa448ca86c44b7ab45da808384a3bd" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 43239 } }
I20260619 08:27:31.305519 12785 ts_tablet_manager.cc:1428] T f0d32be4adff4d7eb25e47533f16fbdb P 73af5c5fdd924807ba6c8e614c0d3b15: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 08:27:31.305522 12787 raft_consensus.cc:374] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:31.305575 12509 heartbeater.cc:503] Master 127.25.127.190:42815 was elected leader, sending a full tablet report...
I20260619 08:27:31.305572 12787 raft_consensus.cc:729] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 559aaf76a95d4e80ab3853d7d20b712a, State: Initialized, Role: FOLLOWER
I20260619 08:27:31.305688 12787 consensus_queue.cc:260] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559aaf76a95d4e80ab3853d7d20b712a" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45619 } } peers { permanent_uuid: "73af5c5fdd924807ba6c8e614c0d3b15" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 42029 } } peers { permanent_uuid: "98aa448ca86c44b7ab45da808384a3bd" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 43239 } }
I20260619 08:27:31.305960 12787 ts_tablet_manager.cc:1428] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 08:27:31.306084 12640 heartbeater.cc:503] Master 127.25.127.190:42815 was elected leader, sending a full tablet report...
I20260619 08:27:31.306324 12786 tablet_bootstrap.cc:492] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd: No bootstrap required, opened a new log
I20260619 08:27:31.306401 12786 ts_tablet_manager.cc:1397] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 08:27:31.308776 12786 raft_consensus.cc:348] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559aaf76a95d4e80ab3853d7d20b712a" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45619 } } peers { permanent_uuid: "73af5c5fdd924807ba6c8e614c0d3b15" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 42029 } } peers { permanent_uuid: "98aa448ca86c44b7ab45da808384a3bd" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 43239 } }
I20260619 08:27:31.308902 12786 raft_consensus.cc:374] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:31.308923 12786 raft_consensus.cc:729] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98aa448ca86c44b7ab45da808384a3bd, State: Initialized, Role: FOLLOWER
I20260619 08:27:31.309059 12786 consensus_queue.cc:260] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559aaf76a95d4e80ab3853d7d20b712a" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45619 } } peers { permanent_uuid: "73af5c5fdd924807ba6c8e614c0d3b15" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 42029 } } peers { permanent_uuid: "98aa448ca86c44b7ab45da808384a3bd" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 43239 } }
I20260619 08:27:31.309312 12786 ts_tablet_manager.cc:1428] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 08:27:31.309368 12771 heartbeater.cc:503] Master 127.25.127.190:42815 was elected leader, sending a full tablet report...
W20260619 08:27:31.400406 12641 tablet.cc:2369] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:27:31.509034 12510 tablet.cc:2369] T f0d32be4adff4d7eb25e47533f16fbdb P 73af5c5fdd924807ba6c8e614c0d3b15: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:27:31.522151 12772 tablet.cc:2369] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:27:31.639200 12793 raft_consensus.cc:482] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:27:31.639333 12793 raft_consensus.cc:504] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559aaf76a95d4e80ab3853d7d20b712a" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45619 } } peers { permanent_uuid: "73af5c5fdd924807ba6c8e614c0d3b15" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 42029 } } peers { permanent_uuid: "98aa448ca86c44b7ab45da808384a3bd" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 43239 } }
I20260619 08:27:31.639793 12793 leader_election.cc:288] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 559aaf76a95d4e80ab3853d7d20b712a (127.25.127.130:45619), 73af5c5fdd924807ba6c8e614c0d3b15 (127.25.127.129:42029)
I20260619 08:27:31.643795 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.640128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:34415 (local address 127.25.127.130:45619)
0619 08:27:31.640195 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:31.640197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:31.640207 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:27:31.640243 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:31.640244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:31.640254 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:31.640357 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:31.640559 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.641308 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.642864 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.643416 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.643495 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:31.643503 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:31.643679 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 08:27:31.643721 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:27:31.643795 12782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.640176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:54363 (local address 127.25.127.129:42029)
0619 08:27:31.640232 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:31.640234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:31.640303 (+ 69us) server_negotiation.cc:407] Connection header received
0619 08:27:31.640331 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:31.640337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:31.640350 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:31.640413 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:31.640553 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.641322 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.642860 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.643417 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.643495 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:31.643503 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:31.643679 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 08:27:31.643721 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 08:27:31.643810 12773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.639964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:45619 (local address 127.25.127.131:34415)
0619 08:27:31.640018 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:31.640031 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:31.640082 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:31.640377 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:31.640378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:31.640379 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:31.640476 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:31.640477 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.641394 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.641395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:31.642761 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:31.642764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.643508 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.643509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:31.643591 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:31.643596 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:27:31.643659 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 08:27:31.643718 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":115}
I20260619 08:27:31.643818 12794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.640029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:42029 (local address 127.25.127.131:54363)
0619 08:27:31.640238 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:31.640246 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:31.640287 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:31.640417 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:31.640418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:31.640419 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:31.640476 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:31.640477 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.641394 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.641395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:31.642761 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:31.642764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.643508 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.643509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:31.643604 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:31.643607 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:27:31.643659 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:27:31.643716 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":10,"thread_start_us":132,"threads_started":1}
I20260619 08:27:31.644315 12594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0d32be4adff4d7eb25e47533f16fbdb" candidate_uuid: "98aa448ca86c44b7ab45da808384a3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "559aaf76a95d4e80ab3853d7d20b712a" is_pre_election: true
I20260619 08:27:31.644500 12594 raft_consensus.cc:2454] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98aa448ca86c44b7ab45da808384a3bd in term 0.
I20260619 08:27:31.644519 12463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0d32be4adff4d7eb25e47533f16fbdb" candidate_uuid: "98aa448ca86c44b7ab45da808384a3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73af5c5fdd924807ba6c8e614c0d3b15" is_pre_election: true
I20260619 08:27:31.644659 12463 raft_consensus.cc:2454] T f0d32be4adff4d7eb25e47533f16fbdb P 73af5c5fdd924807ba6c8e614c0d3b15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98aa448ca86c44b7ab45da808384a3bd in term 0.
I20260619 08:27:31.644801 12661 leader_election.cc:302] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559aaf76a95d4e80ab3853d7d20b712a, 98aa448ca86c44b7ab45da808384a3bd; no voters:
I20260619 08:27:31.644941 12793 raft_consensus.cc:2781] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:27:31.644992 12793 raft_consensus.cc:482] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:27:31.645023 12793 raft_consensus.cc:3037] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:31.645753 12793 raft_consensus.cc:504] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559aaf76a95d4e80ab3853d7d20b712a" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45619 } } peers { permanent_uuid: "73af5c5fdd924807ba6c8e614c0d3b15" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 42029 } } peers { permanent_uuid: "98aa448ca86c44b7ab45da808384a3bd" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 43239 } }
I20260619 08:27:31.645910 12793 leader_election.cc:288] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [CANDIDATE]: Term 1 election: Requested vote from peers 559aaf76a95d4e80ab3853d7d20b712a (127.25.127.130:45619), 73af5c5fdd924807ba6c8e614c0d3b15 (127.25.127.129:42029)
I20260619 08:27:31.646116 12463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0d32be4adff4d7eb25e47533f16fbdb" candidate_uuid: "98aa448ca86c44b7ab45da808384a3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73af5c5fdd924807ba6c8e614c0d3b15"
I20260619 08:27:31.646203 12463 raft_consensus.cc:3037] T f0d32be4adff4d7eb25e47533f16fbdb P 73af5c5fdd924807ba6c8e614c0d3b15 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:31.646976 12463 raft_consensus.cc:2454] T f0d32be4adff4d7eb25e47533f16fbdb P 73af5c5fdd924807ba6c8e614c0d3b15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98aa448ca86c44b7ab45da808384a3bd in term 1.
I20260619 08:27:31.647104 12594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0d32be4adff4d7eb25e47533f16fbdb" candidate_uuid: "98aa448ca86c44b7ab45da808384a3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "559aaf76a95d4e80ab3853d7d20b712a"
I20260619 08:27:31.647157 12659 leader_election.cc:302] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73af5c5fdd924807ba6c8e614c0d3b15, 98aa448ca86c44b7ab45da808384a3bd; no voters:
I20260619 08:27:31.647238 12793 raft_consensus.cc:2781] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:31.647190 12594 raft_consensus.cc:3037] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:31.647413 12793 raft_consensus.cc:686] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [term 1 LEADER]: Becoming Leader. State: Replica: 98aa448ca86c44b7ab45da808384a3bd, State: Running, Role: LEADER
I20260619 08:27:31.647513 12793 consensus_queue.cc:237] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559aaf76a95d4e80ab3853d7d20b712a" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45619 } } peers { permanent_uuid: "73af5c5fdd924807ba6c8e614c0d3b15" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 42029 } } peers { permanent_uuid: "98aa448ca86c44b7ab45da808384a3bd" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 43239 } }
I20260619 08:27:31.648216 12594 raft_consensus.cc:2454] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98aa448ca86c44b7ab45da808384a3bd in term 1.
I20260619 08:27:31.648666 12321 catalog_manager.cc:5495] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 98aa448ca86c44b7ab45da808384a3bd (127.25.127.131). New cstate: current_term: 1 leader_uuid: "98aa448ca86c44b7ab45da808384a3bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559aaf76a95d4e80ab3853d7d20b712a" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73af5c5fdd924807ba6c8e614c0d3b15" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 42029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98aa448ca86c44b7ab45da808384a3bd" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 43239 } health_report { overall_health: HEALTHY } } }
I20260619 08:27:31.737950 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.735609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41582 (local address 127.25.127.131:43239)
0619 08:27:31.735668 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:31.735676 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:31.735683 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:27:31.735720 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:31.735721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:31.735742 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:31.735800 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:27:31.735959 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.736443 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.737206 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:31.737382 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:31.737432 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:31.737601 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:27:31.737692 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:27:31.737764 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:31.737824 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 08:27:31.737832 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 08:27:31.739900 12594 raft_consensus.cc:1261] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a [term 1 FOLLOWER]: Refusing update from remote peer 98aa448ca86c44b7ab45da808384a3bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:27:31.739928 12463 raft_consensus.cc:1261] T f0d32be4adff4d7eb25e47533f16fbdb P 73af5c5fdd924807ba6c8e614c0d3b15 [term 1 FOLLOWER]: Refusing update from remote peer 98aa448ca86c44b7ab45da808384a3bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:27:31.740321 12795 consensus_queue.cc:1035] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "559aaf76a95d4e80ab3853d7d20b712a" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 45619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:27:31.740451 12793 consensus_queue.cc:1035] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "73af5c5fdd924807ba6c8e614c0d3b15" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 42029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:27:31.742936 12798 mvcc.cc:205] Tried to move back new op lower bound from 7298488941523677184 to 7298488941148884992. Current Snapshot: MvccSnapshot[applied={T|T < 7298488941523677184}]
I20260619 08:27:31.743397 12799 mvcc.cc:205] Tried to move back new op lower bound from 7298488941523677184 to 7298488941148884992. Current Snapshot: MvccSnapshot[applied={T|T < 7298488941523677184}]
I20260619 08:27:31.745450 12320 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:27:31.745550 12320 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:27:31.746311 12320 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e6138e4a2141ec86e1d5ec585515a3: Sending DeleteTablet for 3 replicas of tablet f0d32be4adff4d7eb25e47533f16fbdb
I20260619 08:27:31.746640 12574 tablet_service.cc:1514] Processing DeleteTablet for tablet f0d32be4adff4d7eb25e47533f16fbdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:27:31 UTC) from {username='slave'} at 127.0.0.1:46372
I20260619 08:27:31.746728 12443 tablet_service.cc:1514] Processing DeleteTablet for tablet f0d32be4adff4d7eb25e47533f16fbdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:27:31 UTC) from {username='slave'} at 127.0.0.1:39474
I20260619 08:27:31.746894 12808 tablet_replica.cc:330] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a: stopping tablet replica
I20260619 08:27:31.746992 12809 tablet_replica.cc:330] T f0d32be4adff4d7eb25e47533f16fbdb P 73af5c5fdd924807ba6c8e614c0d3b15: stopping tablet replica
I20260619 08:27:31.747004 12808 raft_consensus.cc:2229] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:27:31.747011 12705 tablet_service.cc:1514] Processing DeleteTablet for tablet f0d32be4adff4d7eb25e47533f16fbdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:27:31 UTC) from {username='slave'} at 127.0.0.1:41566
I20260619 08:27:31.747107 12808 raft_consensus.cc:2258] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:27:31.747215 12809 raft_consensus.cc:2229] T f0d32be4adff4d7eb25e47533f16fbdb P 73af5c5fdd924807ba6c8e614c0d3b15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:27:31.747375 12809 raft_consensus.cc:2258] T f0d32be4adff4d7eb25e47533f16fbdb P 73af5c5fdd924807ba6c8e614c0d3b15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:27:31.747480 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 12381
I20260619 08:27:31.747560 12808 ts_tablet_manager.cc:1905] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:27:31.750805 12808 ts_tablet_manager.cc:1918] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:27:31.750895 12808 log.cc:1196] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/f0d32be4adff4d7eb25e47533f16fbdb
I20260619 08:27:31.751169 12808 ts_tablet_manager.cc:1939] T f0d32be4adff4d7eb25e47533f16fbdb P 559aaf76a95d4e80ab3853d7d20b712a: Deleting consensus metadata
I20260619 08:27:31.751572 12308 catalog_manager.cc:4841] TS 559aaf76a95d4e80ab3853d7d20b712a (127.25.127.130:45619): tablet f0d32be4adff4d7eb25e47533f16fbdb (table test-table [id=16b88ccdc2f246cdb9cb25b2308d9386]) successfully deleted
I20260619 08:27:31.752532 12810 tablet_replica.cc:330] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd: stopping tablet replica
I20260619 08:27:31.752652 12810 raft_consensus.cc:2229] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:27:31.752792 12810 raft_consensus.cc:2258] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:27:31.753260 12810 ts_tablet_manager.cc:1905] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 08:27:31.753324 12306 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.127.129:42029 (error 108)
I20260619 08:27:31.753789 12780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:31.753643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:42029 (local address 127.0.0.1:39480)
0619 08:27:31.753706 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:31.753753 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.129:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 08:27:31.753894 12306 catalog_manager.cc:4584] TS 73af5c5fdd924807ba6c8e614c0d3b15 (127.25.127.129:42029): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f0d32be4adff4d7eb25e47533f16fbdb: Network error: Client connection negotiation failed: client connection to 127.25.127.129:42029: connect: Connection refused (error 111)
I20260619 08:27:31.754187 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 12512
I20260619 08:27:31.756213 12810 ts_tablet_manager.cc:1918] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:27:31.756282 12810 log.cc:1196] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/f0d32be4adff4d7eb25e47533f16fbdb
I20260619 08:27:31.756551 12810 ts_tablet_manager.cc:1939] T f0d32be4adff4d7eb25e47533f16fbdb P 98aa448ca86c44b7ab45da808384a3bd: Deleting consensus metadata
I20260619 08:27:31.756984 12306 catalog_manager.cc:4841] TS 98aa448ca86c44b7ab45da808384a3bd (127.25.127.131:43239): tablet f0d32be4adff4d7eb25e47533f16fbdb (table test-table [id=16b88ccdc2f246cdb9cb25b2308d9386]) successfully deleted
I20260619 08:27:31.760600 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 12643
I20260619 08:27:31.765710 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 12290
2026-06-19T08:27:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1058 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:31 dist-test-slave-5rhg krb5kdc[12814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:31 dist-test-slave-5rhg krb5kdc[12814](info): set up 2 sockets
Jun 19 08:27:31 dist-test-slave-5rhg krb5kdc[12814](info): commencing operation
krb5kdc: starting...
W20260619 08:27:32.832971 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:32 dist-test-slave-5rhg krb5kdc[12814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:27:32.948613 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:39861
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:39861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:33.031172 12827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:33.031428 12827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:33.031466 12827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:33.032680 12827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:33.032752 12827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:33.032776 12827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:33.032796 12827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:33.032828 12827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:33.034279 12827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:39861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:39861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:33.034554 12827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:33.034785 12827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:33.039904 12835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:33.039875 12833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:33.039873 12832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:33.039891 12827 server_base.cc:1034] running on GCE node
I20260619 08:27:33.040366 12827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:33.040683 12827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:33.041846 12827 hybrid_clock.cc:648] HybridClock initialized: now 1781857653041829 us; error 44 us; skew 500 ppm
Jun 19 08:27:33 dist-test-slave-5rhg krb5kdc[12814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857653, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:33.044446 12827 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:27:33.044827 12827 webserver.cc:460] Webserver started at http://127.25.127.190:33689/ using document root <none> and password file <none>
I20260619 08:27:33.045094 12827 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:33.045141 12827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:33.045235 12827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:33.046236 12827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "ab1d4af225b74d0a81def5fba3e0f1ae"
format_stamp: "Formatted at 2026-06-19 08:27:33 on dist-test-slave-5rhg"
server_key: "fd1e9b7c74602a2c9ae4b542bb365a51"
server_key_iv: "e751e9d08291618d5dfe8325a16875b2"
server_key_version: "encryptionkey@0"
I20260619 08:27:33.046530 12827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "ab1d4af225b74d0a81def5fba3e0f1ae"
format_stamp: "Formatted at 2026-06-19 08:27:33 on dist-test-slave-5rhg"
server_key: "fd1e9b7c74602a2c9ae4b542bb365a51"
server_key_iv: "e751e9d08291618d5dfe8325a16875b2"
server_key_version: "encryptionkey@0"
I20260619 08:27:33.047793 12827 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:27:33.048527 12842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:33.048707 12827 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:33.048799 12827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "ab1d4af225b74d0a81def5fba3e0f1ae"
format_stamp: "Formatted at 2026-06-19 08:27:33 on dist-test-slave-5rhg"
server_key: "fd1e9b7c74602a2c9ae4b542bb365a51"
server_key_iv: "e751e9d08291618d5dfe8325a16875b2"
server_key_version: "encryptionkey@0"
I20260619 08:27:33.048856 12827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:33.089567 12827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:33.092060 12827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:33.092254 12827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:33.095994 12827 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:39861
I20260619 08:27:33.096086 12894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:39861 every 8 connection(s)
I20260619 08:27:33.096454 12827 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:33.097337 12895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:33.099597 12895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae: Bootstrap starting.
I20260619 08:27:33.100136 12895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:33.100464 12895 log.cc:824] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:33.101128 12895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae: No bootstrap required, opened a new log
I20260619 08:27:33.102654 12895 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d4af225b74d0a81def5fba3e0f1ae" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39861 } }
I20260619 08:27:33.102788 12895 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:33.102811 12895 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab1d4af225b74d0a81def5fba3e0f1ae, State: Initialized, Role: FOLLOWER
I20260619 08:27:33.102912 12895 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d4af225b74d0a81def5fba3e0f1ae" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39861 } }
I20260619 08:27:33.102991 12895 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:33.103029 12895 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:33.103108 12895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:33.103633 12895 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d4af225b74d0a81def5fba3e0f1ae" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39861 } }
I20260619 08:27:33.103750 12895 leader_election.cc:302] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab1d4af225b74d0a81def5fba3e0f1ae; no voters:
I20260619 08:27:33.103965 12895 leader_election.cc:288] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:33.103991 12898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:33.104110 12898 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae [term 1 LEADER]: Becoming Leader. State: Replica: ab1d4af225b74d0a81def5fba3e0f1ae, State: Running, Role: LEADER
I20260619 08:27:33.104103 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 12827
I20260619 08:27:33.104274 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:33.104278 12895 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:33.104277 12898 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d4af225b74d0a81def5fba3e0f1ae" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39861 } }
I20260619 08:27:33.104458 26110 external_mini_cluster.cc:1428] Setting key d734b1565e4a0006b0ce9f68911c707b
I20260619 08:27:33.105028 12900 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab1d4af225b74d0a81def5fba3e0f1ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d4af225b74d0a81def5fba3e0f1ae" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39861 } } }
I20260619 08:27:33.105278 12900 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:33.105587 12899 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab1d4af225b74d0a81def5fba3e0f1ae. Latest consensus state: current_term: 1 leader_uuid: "ab1d4af225b74d0a81def5fba3e0f1ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d4af225b74d0a81def5fba3e0f1ae" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 39861 } } }
I20260619 08:27:33.105741 12899 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae [sys.catalog]: This master's current role is: LEADER
W20260619 08:27:33.106222 12915 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:27:33.106285 12915 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:27:33.106365 12916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:33.106531 12916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:33.108191 12916 catalog_manager.cc:1347] Generated new cluster ID: d94a8fad06774828a56d731459db33eb
I20260619 08:27:33.108242 12916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:27:33 dist-test-slave-5rhg krb5kdc[12814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:27:33.116421 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.104983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40814 (local address 127.25.127.190:39861)
0619 08:27:33.105327 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:33.105339 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:33.105377 (+ 38us) server_negotiation.cc:407] Connection header received
0619 08:27:33.105614 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:33.105622 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:33.105684 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:33.105829 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:33.106759 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.107923 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.109055 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.109655 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.109755 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.112689 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:33.112693 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:33.112694 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:33.112706 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:33.114749 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:33.115160 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:33.115164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:33.115164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:33.115279 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:33.115495 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:33.115496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:33.115496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:33.115793 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:33.115944 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:33.116054 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 08:27:33.116209 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260619 08:27:33.119342 12916 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:33.120087 12916 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:33.125259 12916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae: Generated new TSK 0
I20260619 08:27:33.125496 12916 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.127.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
Entry for principal kudu/127.25.127.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab.
I20260619 08:27:33.158752 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:39861
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:33.243796 12921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:33.244000 12921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:33.244035 12921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:33.245245 12921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:33.245294 12921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:33.245359 12921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:27:33.246850 12921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.129.keytab
--principal=kudu/127.25.127.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:39861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:33.247051 12921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:33.247244 12921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:33.252614 12927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:33.252664 12929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:33.252661 12926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:33.252779 12921 server_base.cc:1034] running on GCE node
I20260619 08:27:33.253111 12921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:33.253376 12921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:33.254552 12921 hybrid_clock.cc:648] HybridClock initialized: now 1781857653254538 us; error 30 us; skew 500 ppm
Jun 19 08:27:33 dist-test-slave-5rhg krb5kdc[12814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857653, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:33.257292 12921 init.cc:377] Logged in from keytab as kudu/127.25.127.129@KRBTEST.COM (short username kudu)
I20260619 08:27:33.257745 12921 webserver.cc:460] Webserver started at http://127.25.127.129:41401/ using document root <none> and password file <none>
I20260619 08:27:33.257972 12921 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:33.258040 12921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:33.258194 12921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:33.259287 12921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "9441a820215d497d85687755e7c4988a"
format_stamp: "Formatted at 2026-06-19 08:27:33 on dist-test-slave-5rhg"
server_key: "83ca7414907e8b96f1803809dadb393b"
server_key_iv: "df515c34420537ec9de1d83bbdfe2c2f"
server_key_version: "encryptionkey@0"
I20260619 08:27:33.259660 12921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "9441a820215d497d85687755e7c4988a"
format_stamp: "Formatted at 2026-06-19 08:27:33 on dist-test-slave-5rhg"
server_key: "83ca7414907e8b96f1803809dadb393b"
server_key_iv: "df515c34420537ec9de1d83bbdfe2c2f"
server_key_version: "encryptionkey@0"
I20260619 08:27:33.261109 12921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:27:33.261864 12936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:33.262046 12921 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:27:33.262153 12921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "9441a820215d497d85687755e7c4988a"
format_stamp: "Formatted at 2026-06-19 08:27:33 on dist-test-slave-5rhg"
server_key: "83ca7414907e8b96f1803809dadb393b"
server_key_iv: "df515c34420537ec9de1d83bbdfe2c2f"
server_key_version: "encryptionkey@0"
I20260619 08:27:33.262230 12921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:33.273306 12921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:33.275775 12921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:33.275950 12921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:33.276268 12921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:33.276605 12921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:33.276661 12921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:33.276705 12921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:33.276737 12921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:33.282580 12921 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:36147
I20260619 08:27:33.282685 13049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:36147 every 8 connection(s)
I20260619 08:27:33.283002 12921 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:27:33.284047 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 12921
I20260619 08:27:33.284240 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:27:33.284462 26110 external_mini_cluster.cc:1428] Setting key a9e05e3eba54a1bcdbaa1223f0f11311
Jun 19 08:27:33 dist-test-slave-5rhg krb5kdc[12814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857653, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.129@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.127.130@KRBTEST.COM; defaulting to no policy
I20260619 08:27:33.293422 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.283942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:37209 (local address 127.25.127.190:39861)
0619 08:27:33.284209 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:33.284212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:33.284267 (+ 55us) server_negotiation.cc:407] Connection header received
0619 08:27:33.284910 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:33.284911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:33.284926 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:33.284971 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:33.286043 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.286556 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.287616 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.287837 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.287927 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.290248 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:33.290249 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:33.290249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:33.290262 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:33.291766 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:33.292239 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:33.292241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:33.292241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:33.292330 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:33.292705 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:33.292706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:33.292706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:33.292842 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:33.292895 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:33.293271 (+ 376us) server_negotiation.cc:299] Negotiation successful
0619 08:27:33.293355 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231}
I20260619 08:27:33.293671 13052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.283870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:39861 (local address 127.25.127.129:37209)
0619 08:27:33.284085 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:33.284149 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:33.284808 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:33.285087 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:33.285092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:33.285721 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:33.285927 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.285942 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.286659 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.286660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.287520 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.287522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.287945 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.287946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.288089 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.288727 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:27:33.288736 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:33.290145 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:33.291872 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:33.291880 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:33.291882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:33.292120 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:33.292420 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:33.292421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:33.292421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:33.292606 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:33.292918 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:33.292926 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:33.293095 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 08:27:33.293190 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 08:27:33.293328 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":108,"threads_started":1}
I20260619 08:27:33.294203 13050 heartbeater.cc:347] Connected to a master server at 127.25.127.190:39861
I20260619 08:27:33.294324 13050 heartbeater.cc:464] Registering TS with master...
I20260619 08:27:33.294582 13050 heartbeater.cc:511] Master 127.25.127.190:39861 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.130@KRBTEST.COM" created.
I20260619 08:27:33.295234 12859 ts_manager.cc:194] Registered new tserver with Master: 9441a820215d497d85687755e7c4988a (127.25.127.129:36147)
I20260619 08:27:33.296239 12859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.129@KRBTEST.COM'} at 127.25.127.129:37209
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
Entry for principal kudu/127.25.127.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab.
I20260619 08:27:33.310348 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:39861
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:33.392578 13055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:33.392838 13055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:33.392866 13055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:33.394101 13055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:33.394143 13055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:33.394169 13055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:27:33.395634 13055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.130.keytab
--principal=kudu/127.25.127.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:39861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:33.395887 13055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:33.396159 13055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:33.401310 13061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:33.401294 13063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:33.401294 13060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:33.401569 13055 server_base.cc:1034] running on GCE node
I20260619 08:27:33.401791 13055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:33.401993 13055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:33.403126 13055 hybrid_clock.cc:648] HybridClock initialized: now 1781857653403115 us; error 29 us; skew 500 ppm
Jun 19 08:27:33 dist-test-slave-5rhg krb5kdc[12814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857653, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:33.405941 13055 init.cc:377] Logged in from keytab as kudu/127.25.127.130@KRBTEST.COM (short username kudu)
I20260619 08:27:33.406400 13055 webserver.cc:460] Webserver started at http://127.25.127.130:39831/ using document root <none> and password file <none>
I20260619 08:27:33.406631 13055 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:33.406701 13055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:33.406857 13055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:33.407924 13055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "467ebbd7da5148e193c1a353adcb5b71"
format_stamp: "Formatted at 2026-06-19 08:27:33 on dist-test-slave-5rhg"
server_key: "f2a1c004faf5e60352bc169a9d1e6bd1"
server_key_iv: "5c25c498488370831839ffe094c0539e"
server_key_version: "encryptionkey@0"
I20260619 08:27:33.408321 13055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "467ebbd7da5148e193c1a353adcb5b71"
format_stamp: "Formatted at 2026-06-19 08:27:33 on dist-test-slave-5rhg"
server_key: "f2a1c004faf5e60352bc169a9d1e6bd1"
server_key_iv: "5c25c498488370831839ffe094c0539e"
server_key_version: "encryptionkey@0"
I20260619 08:27:33.409686 13055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:27:33.410426 13070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:33.410619 13055 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:33.410717 13055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "467ebbd7da5148e193c1a353adcb5b71"
format_stamp: "Formatted at 2026-06-19 08:27:33 on dist-test-slave-5rhg"
server_key: "f2a1c004faf5e60352bc169a9d1e6bd1"
server_key_iv: "5c25c498488370831839ffe094c0539e"
server_key_version: "encryptionkey@0"
I20260619 08:27:33.410809 13055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:33.422850 13055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:33.424813 13055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:33.424971 13055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:33.425232 13055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:33.425544 13055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:33.425596 13055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:33.425643 13055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:33.425674 13055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:33.431674 13055 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:35107
I20260619 08:27:33.431782 13183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:35107 every 8 connection(s)
I20260619 08:27:33.432091 13055 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:27:33.435540 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 13055
I20260619 08:27:33.435642 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:27:33.435851 26110 external_mini_cluster.cc:1428] Setting key d88bea2ed0dfcc2978963cb0b73441fb
Jun 19 08:27:33 dist-test-slave-5rhg krb5kdc[12814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857653, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.130@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:27:33.442054 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.432922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:33195 (local address 127.25.127.190:39861)
0619 08:27:33.432993 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:33.432995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:33.433221 (+ 226us) server_negotiation.cc:407] Connection header received
0619 08:27:33.433660 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:33.433661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:33.433672 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:33.433741 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:33.434665 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.435173 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.436436 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.436743 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.436816 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.438986 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:33.438988 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:33.438988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:33.438996 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:33.440620 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:33.441011 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:33.441012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:33.441012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:33.441088 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:33.441316 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:33.441317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:33.441317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:33.441491 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:33.441585 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:33.441816 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 08:27:33.441915 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 08:27:33.442063 13186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.432866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:39861 (local address 127.25.127.130:33195)
0619 08:27:33.433031 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:33.433084 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:33.433569 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:33.433798 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:33.433801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:33.434371 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:33.434575 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.434580 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.435258 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.435259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.436363 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.436366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.436830 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.436831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.436939 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.437509 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:27:33.437518 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:33.438888 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:33.440700 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:33.440706 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:33.440708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:33.440931 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:33.441155 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:33.441156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:33.441156 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:33.441252 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:33.441567 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:33.441571 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:33.441684 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 08:27:33.441766 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 08:27:33.441892 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":96,"threads_started":1}
I20260619 08:27:33.442606 13184 heartbeater.cc:347] Connected to a master server at 127.25.127.190:39861
I20260619 08:27:33.442718 13184 heartbeater.cc:464] Registering TS with master...
I20260619 08:27:33.442921 13184 heartbeater.cc:511] Master 127.25.127.190:39861 requested a full tablet report, sending...
I20260619 08:27:33.443341 12859 ts_manager.cc:194] Registered new tserver with Master: 467ebbd7da5148e193c1a353adcb5b71 (127.25.127.130:35107)
I20260619 08:27:33.443719 12859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.130@KRBTEST.COM'} at 127.25.127.130:33195
WARNING: no policy specified for kudu/127.25.127.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
Entry for principal kudu/127.25.127.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab.
I20260619 08:27:33.461187 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:39861
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:33.544188 13189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:33.544396 13189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:33.544420 13189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:33.545491 13189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:33.545526 13189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:33.545549 13189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:27:33.547060 13189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.131.keytab
--principal=kudu/127.25.127.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:39861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:33.547258 13189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:33.547435 13189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:33.552603 13194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:33.552590 13197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:33.552609 13195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:33.552968 13189 server_base.cc:1034] running on GCE node
I20260619 08:27:33.553184 13189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:33.553390 13189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:33.554545 13189 hybrid_clock.cc:648] HybridClock initialized: now 1781857653554516 us; error 44 us; skew 500 ppm
Jun 19 08:27:33 dist-test-slave-5rhg krb5kdc[12814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857653, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:33.557364 13189 init.cc:377] Logged in from keytab as kudu/127.25.127.131@KRBTEST.COM (short username kudu)
I20260619 08:27:33.557827 13189 webserver.cc:460] Webserver started at http://127.25.127.131:43753/ using document root <none> and password file <none>
I20260619 08:27:33.558063 13189 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:33.558112 13189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:33.558288 13189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:33.559401 13189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "33d5cc696f5e4cd28f4453b848f7c8f0"
format_stamp: "Formatted at 2026-06-19 08:27:33 on dist-test-slave-5rhg"
server_key: "e97d202d2a0381f15e9cf33c834cbc0a"
server_key_iv: "abdace300268248dfae26d5e39d095c6"
server_key_version: "encryptionkey@0"
I20260619 08:27:33.559803 13189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "33d5cc696f5e4cd28f4453b848f7c8f0"
format_stamp: "Formatted at 2026-06-19 08:27:33 on dist-test-slave-5rhg"
server_key: "e97d202d2a0381f15e9cf33c834cbc0a"
server_key_iv: "abdace300268248dfae26d5e39d095c6"
server_key_version: "encryptionkey@0"
I20260619 08:27:33.561285 13189 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:27:33.562059 13204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:33.562261 13189 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:27:33.562361 13189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "33d5cc696f5e4cd28f4453b848f7c8f0"
format_stamp: "Formatted at 2026-06-19 08:27:33 on dist-test-slave-5rhg"
server_key: "e97d202d2a0381f15e9cf33c834cbc0a"
server_key_iv: "abdace300268248dfae26d5e39d095c6"
server_key_version: "encryptionkey@0"
I20260619 08:27:33.562443 13189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:33.587049 13189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:33.589615 13189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:33.589785 13189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:33.590044 13189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:33.590369 13189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:33.590422 13189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:33.590457 13189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:33.590487 13189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:33.596485 13189 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:44303
I20260619 08:27:33.596570 13317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:44303 every 8 connection(s)
I20260619 08:27:33.596905 13189 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:27:33 dist-test-slave-5rhg krb5kdc[12814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857653, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.131@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:27:33.606251 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.597708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:55739 (local address 127.25.127.190:39861)
0619 08:27:33.597765 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:33.597767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:33.598087 (+ 320us) server_negotiation.cc:407] Connection header received
0619 08:27:33.598468 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:33.598470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:33.598482 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:33.598560 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:33.599478 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.599971 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.601014 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.601213 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.601269 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.603321 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:33.603322 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:33.603323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:33.603329 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:33.604828 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:33.605222 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:33.605223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:33.605223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:33.605276 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:33.605505 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:33.605506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:33.605506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:33.605635 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:33.605709 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:33.606049 (+ 340us) server_negotiation.cc:299] Negotiation successful
0619 08:27:33.606117 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 08:27:33.606314 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 13189
I20260619 08:27:33.606396 13320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.597689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:39861 (local address 127.25.127.131:55739)
0619 08:27:33.597843 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:33.597885 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:33.598380 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:33.598618 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:33.598621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:33.599203 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:33.599402 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.599407 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.600048 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.600049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.600935 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.600937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.601289 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.601290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.601391 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.601843 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:27:33.601851 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:33.603237 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:33.604894 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:33.604900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:33.604901 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:33.605163 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:33.605332 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:33.605333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:33.605333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:33.605445 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:33.605728 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:33.605734 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:33.605902 (+ 168us) client_negotiation.cc:769] Sending connection context
0619 08:27:33.606018 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 08:27:33.606193 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260619 08:27:33.606431 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:27:33.606640 26110 external_mini_cluster.cc:1428] Setting key c3570a070029abdb74b6d916a9669620
I20260619 08:27:33.606956 13318 heartbeater.cc:347] Connected to a master server at 127.25.127.190:39861
I20260619 08:27:33.607072 13318 heartbeater.cc:464] Registering TS with master...
I20260619 08:27:33.607271 13318 heartbeater.cc:511] Master 127.25.127.190:39861 requested a full tablet report, sending...
I20260619 08:27:33.607674 12859 ts_manager.cc:194] Registered new tserver with Master: 33d5cc696f5e4cd28f4453b848f7c8f0 (127.25.127.131:44303)
I20260619 08:27:33.608016 12859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.127.131@KRBTEST.COM'} at 127.25.127.131:55739
I20260619 08:27:33.618530 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:27:33.625860 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.619813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40830 (local address 127.25.127.190:39861)
0619 08:27:33.619879 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:33.619881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:33.619990 (+ 109us) server_negotiation.cc:407] Connection header received
0619 08:27:33.620031 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:33.620032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:33.620041 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:33.620079 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:33.620666 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.621154 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.621857 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.622043 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.622095 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.623279 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:33.623280 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:33.623281 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:33.623286 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:33.624758 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:33.625074 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:33.625075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:33.625075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:33.625132 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:33.625369 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:33.625369 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:33.625369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:33.625520 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:33.625606 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:33.625732 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 08:27:33.625791 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:27:33.627457 12859 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:27:33.629221 12859 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:27:33.635651 13329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.631586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:35107 (local address 127.0.0.1:47520)
0619 08:27:33.631820 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:33.631830 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:33.631870 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:33.632269 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:33.632272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:33.632275 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:33.632347 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.632348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.633305 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.633306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.634435 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.634437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.635325 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.635326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.635421 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.635436 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 08:27:33.635514 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 08:27:33.635565 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":90,"threads_started":1}
I20260619 08:27:33.635691 13334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.631848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47520 (local address 127.25.127.130:35107)
0619 08:27:33.632026 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:33.632031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:33.632041 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:27:33.632101 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:33.632103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:33.632115 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:33.632258 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:33.632424 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.633204 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.634568 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.635236 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.635313 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.635371 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:33.635548 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 08:27:33.635598 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
I20260619 08:27:33.636014 13331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.631586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:44303 (local address 127.0.0.1:58016)
0619 08:27:33.631946 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:33.631956 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:33.631994 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:33.632361 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:33.632362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:33.632365 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:33.632420 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.632422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.633766 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.633767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.634788 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.634790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.635746 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.635747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.635820 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.635824 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:27:33.635905 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 08:27:33.635947 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":6,"thread_start_us":82,"threads_started":1}
I20260619 08:27:33.636045 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.631820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58016 (local address 127.25.127.131:44303)
0619 08:27:33.632007 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:33.632013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:33.632026 (+ 13us) server_negotiation.cc:407] Connection header received
0619 08:27:33.632071 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:33.632073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:33.632091 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:33.632276 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:33.632506 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.633669 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.634880 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.635588 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.635659 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.635709 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:33.635922 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 08:27:33.635967 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":78,"threads_started":1}
I20260619 08:27:33.636829 13118 tablet_service.cc:1467] Processing CreateTablet for tablet d7a49c67fab941ee9c0ab97636fbed34 (DEFAULT_TABLE table=test-table [id=5077b78bd2d343b188b929d6d07d2c89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:27:33.636909 13252 tablet_service.cc:1467] Processing CreateTablet for tablet d7a49c67fab941ee9c0ab97636fbed34 (DEFAULT_TABLE table=test-table [id=5077b78bd2d343b188b929d6d07d2c89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:27:33.637231 13330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.631542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:36147 (local address 127.0.0.1:51462)
0619 08:27:33.631735 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:33.631774 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:33.631844 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:33.632399 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:33.632400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:33.632401 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:33.632455 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.632457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.634874 (+ 2417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.634874 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.635732 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.635735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.636976 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.636977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.637061 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.637066 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:27:33.637115 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 08:27:33.637155 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":2,"thread_start_us":76,"threads_started":1}
I20260619 08:27:33.637236 13252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7a49c67fab941ee9c0ab97636fbed34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:33.637238 13118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7a49c67fab941ee9c0ab97636fbed34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:33.637370 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.631654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51462 (local address 127.25.127.129:36147)
0619 08:27:33.632082 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:33.632086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:33.632096 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:27:33.632132 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:33.632134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:33.632174 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:33.632314 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:33.632536 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.633654 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.635837 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.636609 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.636687 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.636736 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:33.637151 (+ 415us) server_negotiation.cc:299] Negotiation successful
0619 08:27:33.637204 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":78,"threads_started":1}
I20260619 08:27:33.637987 12984 tablet_service.cc:1467] Processing CreateTablet for tablet d7a49c67fab941ee9c0ab97636fbed34 (DEFAULT_TABLE table=test-table [id=5077b78bd2d343b188b929d6d07d2c89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:27:33.638309 12984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7a49c67fab941ee9c0ab97636fbed34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:33.639789 13335 tablet_bootstrap.cc:492] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0: Bootstrap starting.
I20260619 08:27:33.640475 13335 tablet_bootstrap.cc:654] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:33.640661 13336 tablet_bootstrap.cc:492] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71: Bootstrap starting.
I20260619 08:27:33.641084 13335 log.cc:824] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:33.641310 13336 tablet_bootstrap.cc:654] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:33.641865 13335 tablet_bootstrap.cc:492] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0: No bootstrap required, opened a new log
I20260619 08:27:33.641876 13336 log.cc:824] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:33.641902 13338 tablet_bootstrap.cc:492] T d7a49c67fab941ee9c0ab97636fbed34 P 9441a820215d497d85687755e7c4988a: Bootstrap starting.
I20260619 08:27:33.641938 13335 ts_tablet_manager.cc:1397] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:27:33.642508 13338 tablet_bootstrap.cc:654] T d7a49c67fab941ee9c0ab97636fbed34 P 9441a820215d497d85687755e7c4988a: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:33.642668 13336 tablet_bootstrap.cc:492] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71: No bootstrap required, opened a new log
I20260619 08:27:33.642746 13336 ts_tablet_manager.cc:1397] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:27:33.643087 13338 log.cc:824] T d7a49c67fab941ee9c0ab97636fbed34 P 9441a820215d497d85687755e7c4988a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:33.643846 13338 tablet_bootstrap.cc:492] T d7a49c67fab941ee9c0ab97636fbed34 P 9441a820215d497d85687755e7c4988a: No bootstrap required, opened a new log
I20260619 08:27:33.643913 13338 ts_tablet_manager.cc:1397] T d7a49c67fab941ee9c0ab97636fbed34 P 9441a820215d497d85687755e7c4988a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:27:33.643818 13335 raft_consensus.cc:348] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441a820215d497d85687755e7c4988a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36147 } } peers { permanent_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 44303 } } peers { permanent_uuid: "467ebbd7da5148e193c1a353adcb5b71" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35107 } }
I20260619 08:27:33.644006 13335 raft_consensus.cc:374] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:33.644048 13335 raft_consensus.cc:729] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d5cc696f5e4cd28f4453b848f7c8f0, State: Initialized, Role: FOLLOWER
I20260619 08:27:33.644186 13335 consensus_queue.cc:260] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441a820215d497d85687755e7c4988a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36147 } } peers { permanent_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 44303 } } peers { permanent_uuid: "467ebbd7da5148e193c1a353adcb5b71" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35107 } }
I20260619 08:27:33.644479 13335 ts_tablet_manager.cc:1428] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:27:33.644618 13318 heartbeater.cc:503] Master 127.25.127.190:39861 was elected leader, sending a full tablet report...
I20260619 08:27:33.644851 13336 raft_consensus.cc:348] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441a820215d497d85687755e7c4988a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36147 } } peers { permanent_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 44303 } } peers { permanent_uuid: "467ebbd7da5148e193c1a353adcb5b71" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35107 } }
I20260619 08:27:33.645025 13336 raft_consensus.cc:374] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:33.645061 13336 raft_consensus.cc:729] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 467ebbd7da5148e193c1a353adcb5b71, State: Initialized, Role: FOLLOWER
I20260619 08:27:33.645195 13336 consensus_queue.cc:260] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441a820215d497d85687755e7c4988a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36147 } } peers { permanent_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 44303 } } peers { permanent_uuid: "467ebbd7da5148e193c1a353adcb5b71" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35107 } }
I20260619 08:27:33.645462 13336 ts_tablet_manager.cc:1428] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:27:33.645583 13184 heartbeater.cc:503] Master 127.25.127.190:39861 was elected leader, sending a full tablet report...
I20260619 08:27:33.645941 13338 raft_consensus.cc:348] T d7a49c67fab941ee9c0ab97636fbed34 P 9441a820215d497d85687755e7c4988a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441a820215d497d85687755e7c4988a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36147 } } peers { permanent_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 44303 } } peers { permanent_uuid: "467ebbd7da5148e193c1a353adcb5b71" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35107 } }
I20260619 08:27:33.646152 13338 raft_consensus.cc:374] T d7a49c67fab941ee9c0ab97636fbed34 P 9441a820215d497d85687755e7c4988a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:33.646201 13338 raft_consensus.cc:729] T d7a49c67fab941ee9c0ab97636fbed34 P 9441a820215d497d85687755e7c4988a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9441a820215d497d85687755e7c4988a, State: Initialized, Role: FOLLOWER
I20260619 08:27:33.646347 13338 consensus_queue.cc:260] T d7a49c67fab941ee9c0ab97636fbed34 P 9441a820215d497d85687755e7c4988a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441a820215d497d85687755e7c4988a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36147 } } peers { permanent_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 44303 } } peers { permanent_uuid: "467ebbd7da5148e193c1a353adcb5b71" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35107 } }
I20260619 08:27:33.646641 13338 ts_tablet_manager.cc:1428] T d7a49c67fab941ee9c0ab97636fbed34 P 9441a820215d497d85687755e7c4988a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:27:33.646670 13050 heartbeater.cc:503] Master 127.25.127.190:39861 was elected leader, sending a full tablet report...
W20260619 08:27:33.682711 13185 tablet.cc:2369] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:27:33.704619 13342 raft_consensus.cc:482] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:27:33.704751 13342 raft_consensus.cc:504] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441a820215d497d85687755e7c4988a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36147 } } peers { permanent_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 44303 } } peers { permanent_uuid: "467ebbd7da5148e193c1a353adcb5b71" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35107 } }
I20260619 08:27:33.705196 13342 leader_election.cc:288] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9441a820215d497d85687755e7c4988a (127.25.127.129:36147), 33d5cc696f5e4cd28f4453b848f7c8f0 (127.25.127.131:44303)
I20260619 08:27:33.706729 13341 raft_consensus.cc:482] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:27:33.706947 13341 raft_consensus.cc:504] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441a820215d497d85687755e7c4988a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36147 } } peers { permanent_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 44303 } } peers { permanent_uuid: "467ebbd7da5148e193c1a353adcb5b71" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35107 } }
I20260619 08:27:33.707309 13341 leader_election.cc:288] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9441a820215d497d85687755e7c4988a (127.25.127.129:36147), 467ebbd7da5148e193c1a353adcb5b71 (127.25.127.130:35107)
I20260619 08:27:33.709017 13344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.705402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:36147 (local address 127.25.127.130:52171)
0619 08:27:33.705591 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:33.705603 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:33.705688 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:33.705843 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:33.705844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:33.705845 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:33.705921 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.705922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.706831 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.706832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.707931 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.707934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.708724 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.708725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.708812 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.708816 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:27:33.708861 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 08:27:33.708901 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":128,"threads_started":1}
I20260619 08:27:33.709167 13186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.705379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:44303 (local address 127.25.127.130:43937)
0619 08:27:33.705434 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:33.705447 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:33.705492 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:33.705669 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:33.705670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:33.705672 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:33.705749 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.705750 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.706708 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.706709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.707931 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.707934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.708889 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.708890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.708990 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.708993 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:27:33.709064 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 08:27:33.709100 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":5}
I20260619 08:27:33.709435 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.705469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:43937 (local address 127.25.127.131:44303)
0619 08:27:33.705511 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:33.705513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:33.705544 (+ 31us) server_negotiation.cc:407] Connection header received
0619 08:27:33.705582 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:33.705582 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:33.705590 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:33.705652 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:33.705839 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.706602 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.708208 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.708790 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.708853 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.708860 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:33.709330 (+ 470us) server_negotiation.cc:299] Negotiation successful
0619 08:27:33.709373 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:27:33.709559 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.705451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:52171 (local address 127.25.127.129:36147)
0619 08:27:33.705499 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:33.705501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:33.705674 (+ 173us) server_negotiation.cc:407] Connection header received
0619 08:27:33.705757 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:33.705758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:33.705767 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:33.705841 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:33.706042 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.706750 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.708018 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.708613 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.708680 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.708690 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:33.709452 (+ 762us) server_negotiation.cc:299] Negotiation successful
0619 08:27:33.709494 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:27:33.709848 13004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a49c67fab941ee9c0ab97636fbed34" candidate_uuid: "467ebbd7da5148e193c1a353adcb5b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9441a820215d497d85687755e7c4988a" is_pre_election: true
I20260619 08:27:33.709995 13004 raft_consensus.cc:2454] T d7a49c67fab941ee9c0ab97636fbed34 P 9441a820215d497d85687755e7c4988a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 467ebbd7da5148e193c1a353adcb5b71 in term 0.
I20260619 08:27:33.710210 13074 leader_election.cc:302] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 467ebbd7da5148e193c1a353adcb5b71, 9441a820215d497d85687755e7c4988a; no voters:
I20260619 08:27:33.710191 13272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a49c67fab941ee9c0ab97636fbed34" candidate_uuid: "467ebbd7da5148e193c1a353adcb5b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0" is_pre_election: true
I20260619 08:27:33.710300 13272 raft_consensus.cc:2454] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 467ebbd7da5148e193c1a353adcb5b71 in term 0.
I20260619 08:27:33.710332 13342 raft_consensus.cc:2781] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:27:33.710376 13342 raft_consensus.cc:482] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:27:33.710403 13342 raft_consensus.cc:3037] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:33.711239 13334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.708085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:41663 (local address 127.25.127.130:35107)
0619 08:27:33.708125 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:33.708127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:33.708222 (+ 95us) server_negotiation.cc:407] Connection header received
0619 08:27:33.708313 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:33.708314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:33.708322 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:33.708350 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:33.708535 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.709203 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.710476 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.710885 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.710921 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.710926 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:33.711131 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 08:27:33.711167 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:27:33.711342 13320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.707407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:36147 (local address 127.25.127.131:60765)
0619 08:27:33.707454 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:33.707463 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:33.707500 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:33.707752 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:33.707753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:33.707755 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:33.707816 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.707817 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.708782 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.708783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.709990 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.709993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.711100 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.711101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.711166 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.711169 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:27:33.711222 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 08:27:33.711263 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 08:27:33.711385 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.707460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:60765 (local address 127.25.127.129:36147)
0619 08:27:33.707614 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:33.707616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:33.707623 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:27:33.707645 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:33.707646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:33.707655 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:33.707884 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:33.707913 (+ 29us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.708681 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.710430 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.711001 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.711095 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.711104 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:33.711243 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 08:27:33.711290 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
I20260619 08:27:33.711460 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.707690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:35107 (local address 127.25.127.131:41663)
0619 08:27:33.708167 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:33.708177 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:33.708245 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:33.708403 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:33.708403 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:33.708404 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:33.708459 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.708460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.709314 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.709315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.710351 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:33.710353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.710973 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.710973 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:33.711029 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.711031 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:27:33.711336 (+ 305us) client_negotiation.cc:240] Negotiation successful
0619 08:27:33.711374 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":76,"threads_started":1}
I20260619 08:27:33.711431 13342 raft_consensus.cc:504] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441a820215d497d85687755e7c4988a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36147 } } peers { permanent_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 44303 } } peers { permanent_uuid: "467ebbd7da5148e193c1a353adcb5b71" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35107 } }
I20260619 08:27:33.711560 13342 leader_election.cc:288] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [CANDIDATE]: Term 1 election: Requested vote from peers 9441a820215d497d85687755e7c4988a (127.25.127.129:36147), 33d5cc696f5e4cd28f4453b848f7c8f0 (127.25.127.131:44303)
I20260619 08:27:33.711547 13004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a49c67fab941ee9c0ab97636fbed34" candidate_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9441a820215d497d85687755e7c4988a" is_pre_election: true
I20260619 08:27:33.711625 13004 raft_consensus.cc:2454] T d7a49c67fab941ee9c0ab97636fbed34 P 9441a820215d497d85687755e7c4988a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d5cc696f5e4cd28f4453b848f7c8f0 in term 0.
I20260619 08:27:33.711691 13003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a49c67fab941ee9c0ab97636fbed34" candidate_uuid: "467ebbd7da5148e193c1a353adcb5b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9441a820215d497d85687755e7c4988a"
I20260619 08:27:33.711711 13138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a49c67fab941ee9c0ab97636fbed34" candidate_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "467ebbd7da5148e193c1a353adcb5b71" is_pre_election: true
I20260619 08:27:33.711817 13138 raft_consensus.cc:2379] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d5cc696f5e4cd28f4453b848f7c8f0 in current term 1: Already voted for candidate 467ebbd7da5148e193c1a353adcb5b71 in this term.
I20260619 08:27:33.711774 13003 raft_consensus.cc:3037] T d7a49c67fab941ee9c0ab97636fbed34 P 9441a820215d497d85687755e7c4988a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:33.711831 13208 leader_election.cc:302] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d5cc696f5e4cd28f4453b848f7c8f0, 9441a820215d497d85687755e7c4988a; no voters:
I20260619 08:27:33.711956 13341 raft_consensus.cc:2781] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:27:33.712002 13341 raft_consensus.cc:482] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:27:33.712037 13341 raft_consensus.cc:3037] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:33.712746 13003 raft_consensus.cc:2454] T d7a49c67fab941ee9c0ab97636fbed34 P 9441a820215d497d85687755e7c4988a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 467ebbd7da5148e193c1a353adcb5b71 in term 1.
I20260619 08:27:33.712796 13341 raft_consensus.cc:504] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441a820215d497d85687755e7c4988a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36147 } } peers { permanent_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 44303 } } peers { permanent_uuid: "467ebbd7da5148e193c1a353adcb5b71" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35107 } }
I20260619 08:27:33.712872 13272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a49c67fab941ee9c0ab97636fbed34" candidate_uuid: "467ebbd7da5148e193c1a353adcb5b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0"
I20260619 08:27:33.712965 13272 raft_consensus.cc:2379] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 467ebbd7da5148e193c1a353adcb5b71 in current term 1: Already voted for candidate 33d5cc696f5e4cd28f4453b848f7c8f0 in this term.
I20260619 08:27:33.712949 13341 leader_election.cc:288] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [CANDIDATE]: Term 1 election: Requested vote from peers 9441a820215d497d85687755e7c4988a (127.25.127.129:36147), 467ebbd7da5148e193c1a353adcb5b71 (127.25.127.130:35107)
I20260619 08:27:33.713124 13074 leader_election.cc:302] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 467ebbd7da5148e193c1a353adcb5b71, 9441a820215d497d85687755e7c4988a; no voters:
I20260619 08:27:33.713124 13138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a49c67fab941ee9c0ab97636fbed34" candidate_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "467ebbd7da5148e193c1a353adcb5b71"
I20260619 08:27:33.713222 13342 raft_consensus.cc:2781] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:33.713265 13003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7a49c67fab941ee9c0ab97636fbed34" candidate_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9441a820215d497d85687755e7c4988a"
I20260619 08:27:33.713354 13003 raft_consensus.cc:2379] T d7a49c67fab941ee9c0ab97636fbed34 P 9441a820215d497d85687755e7c4988a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d5cc696f5e4cd28f4453b848f7c8f0 in current term 1: Already voted for candidate 467ebbd7da5148e193c1a353adcb5b71 in this term.
I20260619 08:27:33.713388 13342 raft_consensus.cc:686] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [term 1 LEADER]: Becoming Leader. State: Replica: 467ebbd7da5148e193c1a353adcb5b71, State: Running, Role: LEADER
I20260619 08:27:33.713469 13342 consensus_queue.cc:237] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441a820215d497d85687755e7c4988a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36147 } } peers { permanent_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 44303 } } peers { permanent_uuid: "467ebbd7da5148e193c1a353adcb5b71" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35107 } }
I20260619 08:27:33.714092 13205 leader_election.cc:302] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d5cc696f5e4cd28f4453b848f7c8f0; no voters: 467ebbd7da5148e193c1a353adcb5b71, 9441a820215d497d85687755e7c4988a
I20260619 08:27:33.714188 13341 raft_consensus.cc:2726] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260619 08:27:33.714334 12859 catalog_manager.cc:5495] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 reported cstate change: term changed from 0 to 1, leader changed from <none> to 467ebbd7da5148e193c1a353adcb5b71 (127.25.127.130). New cstate: current_term: 1 leader_uuid: "467ebbd7da5148e193c1a353adcb5b71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9441a820215d497d85687755e7c4988a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 44303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "467ebbd7da5148e193c1a353adcb5b71" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 35107 } health_report { overall_health: HEALTHY } } }
I20260619 08:27:33.729151 13334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.726837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47534 (local address 127.25.127.130:35107)
0619 08:27:33.726904 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:33.726905 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:33.726913 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:27:33.726946 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:33.726946 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:33.726963 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:33.727024 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:27:33.727166 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.727632 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.728411 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.728601 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.728653 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.728776 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:27:33.728872 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:27:33.728927 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:33.729061 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 08:27:33.729079 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 08:27:33.730859 13003 raft_consensus.cc:1261] T d7a49c67fab941ee9c0ab97636fbed34 P 9441a820215d497d85687755e7c4988a [term 1 FOLLOWER]: Refusing update from remote peer 467ebbd7da5148e193c1a353adcb5b71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:27:33.730872 13272 raft_consensus.cc:1261] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [term 1 FOLLOWER]: Refusing update from remote peer 467ebbd7da5148e193c1a353adcb5b71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:27:33.731176 13342 consensus_queue.cc:1035] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9441a820215d497d85687755e7c4988a" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 36147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:27:33.731340 13347 consensus_queue.cc:1035] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d5cc696f5e4cd28f4453b848f7c8f0" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:27:33.734678 13351 mvcc.cc:205] Tried to move back new op lower bound from 7298488949679054848 to 7298488949610983424. Current Snapshot: MvccSnapshot[applied={T|T < 7298488949679054848}]
I20260619 08:27:33.736900 12859 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:27:33.736984 12859 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:27:33.737800 12859 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab1d4af225b74d0a81def5fba3e0f1ae: Sending DeleteTablet for 3 replicas of tablet d7a49c67fab941ee9c0ab97636fbed34
I20260619 08:27:33.738179 13118 tablet_service.cc:1514] Processing DeleteTablet for tablet d7a49c67fab941ee9c0ab97636fbed34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:27:33 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:47520
I20260619 08:27:33.738195 12984 tablet_service.cc:1514] Processing DeleteTablet for tablet d7a49c67fab941ee9c0ab97636fbed34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:27:33 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:51462
I20260619 08:27:33.738430 13359 tablet_replica.cc:330] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71: stopping tablet replica
I20260619 08:27:33.738412 13252 tablet_service.cc:1514] Processing DeleteTablet for tablet d7a49c67fab941ee9c0ab97636fbed34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:27:33 UTC) from {username='kudu', principal='kudu/127.25.127.190@KRBTEST.COM'} at 127.0.0.1:58016
I20260619 08:27:33.738523 13359 raft_consensus.cc:2229] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:27:33.738631 13359 raft_consensus.cc:2258] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:27:33.738703 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 12921
I20260619 08:27:33.738843 13361 tablet_replica.cc:330] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0: stopping tablet replica
I20260619 08:27:33.738965 13361 raft_consensus.cc:2229] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:27:33.739104 13361 raft_consensus.cc:2258] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:27:33.739555 13361 ts_tablet_manager.cc:1905] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:27:33.740185 13359 ts_tablet_manager.cc:1905] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:27:33.741824 13361 ts_tablet_manager.cc:1918] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:27:33.741967 13361 log.cc:1196] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/d7a49c67fab941ee9c0ab97636fbed34
I20260619 08:27:33.742136 13359 ts_tablet_manager.cc:1918] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:27:33.742215 13359 log.cc:1196] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/d7a49c67fab941ee9c0ab97636fbed34
I20260619 08:27:33.742450 13361 ts_tablet_manager.cc:1939] T d7a49c67fab941ee9c0ab97636fbed34 P 33d5cc696f5e4cd28f4453b848f7c8f0: Deleting consensus metadata
I20260619 08:27:33.742482 13359 ts_tablet_manager.cc:1939] T d7a49c67fab941ee9c0ab97636fbed34 P 467ebbd7da5148e193c1a353adcb5b71: Deleting consensus metadata
I20260619 08:27:33.742883 12845 catalog_manager.cc:4841] TS 33d5cc696f5e4cd28f4453b848f7c8f0 (127.25.127.131:44303): tablet d7a49c67fab941ee9c0ab97636fbed34 (table test-table [id=5077b78bd2d343b188b929d6d07d2c89]) successfully deleted
I20260619 08:27:33.742880 12843 catalog_manager.cc:4841] TS 467ebbd7da5148e193c1a353adcb5b71 (127.25.127.130:35107): tablet d7a49c67fab941ee9c0ab97636fbed34 (table test-table [id=5077b78bd2d343b188b929d6d07d2c89]) successfully deleted
W20260619 08:27:33.746093 12846 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.127.129:36147 (error 108)
I20260619 08:27:33.746414 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 13055
I20260619 08:27:33.746624 13329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.746496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:36147 (local address 127.0.0.1:51468)
0619 08:27:33.746547 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:33.746583 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.129:36147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 08:27:33.746744 12846 catalog_manager.cc:4584] TS 9441a820215d497d85687755e7c4988a (127.25.127.129:36147): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d7a49c67fab941ee9c0ab97636fbed34: Network error: Client connection negotiation failed: client connection to 127.25.127.129:36147: connect: Connection refused (error 111)
I20260619 08:27:33.751756 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 13189
I20260619 08:27:33.756506 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 12827
2026-06-19T08:27:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1992 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T08:27:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:33Z Disabled control of system clock
I20260619 08:27:33.784552 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:46695
--webserver_interface=127.25.127.190
--webserver_port=0
--builtin_ntp_servers=127.25.127.148:42489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:46695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:27:33.865880 13366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:33.866118 13366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:33.866140 13366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:33.867276 13366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:33.867313 13366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:33.867326 13366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:33.867339 13366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:33.867354 13366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:33.868739 13366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:46695
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:46695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:33.868958 13366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:33.869212 13366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:33.874159 13372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:33.874172 13371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:33.874172 13374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:33.874480 13366 server_base.cc:1034] running on GCE node
I20260619 08:27:33.874760 13366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:33.875080 13366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:33.876221 13366 hybrid_clock.cc:648] HybridClock initialized: now 1781857653876201 us; error 43 us; skew 500 ppm
I20260619 08:27:33.877614 13366 webserver.cc:460] Webserver started at http://127.25.127.190:42559/ using document root <none> and password file <none>
I20260619 08:27:33.877873 13366 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:33.877983 13366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:33.878136 13366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:33.879242 13366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "bce17afa27374293a037e52e1afd8412"
format_stamp: "Formatted at 2026-06-19 08:27:33 on dist-test-slave-5rhg"
server_key: "b15c0953d14ad86b1613af475f928ea7"
server_key_iv: "59b8d04362a106d03e94e70e94ce2eeb"
server_key_version: "encryptionkey@0"
I20260619 08:27:33.879613 13366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "bce17afa27374293a037e52e1afd8412"
format_stamp: "Formatted at 2026-06-19 08:27:33 on dist-test-slave-5rhg"
server_key: "b15c0953d14ad86b1613af475f928ea7"
server_key_iv: "59b8d04362a106d03e94e70e94ce2eeb"
server_key_version: "encryptionkey@0"
I20260619 08:27:33.881040 13366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:27:33.881794 13380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:33.881979 13366 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:27:33.882084 13366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "bce17afa27374293a037e52e1afd8412"
format_stamp: "Formatted at 2026-06-19 08:27:33 on dist-test-slave-5rhg"
server_key: "b15c0953d14ad86b1613af475f928ea7"
server_key_iv: "59b8d04362a106d03e94e70e94ce2eeb"
server_key_version: "encryptionkey@0"
I20260619 08:27:33.882159 13366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:33.899324 13366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:33.899624 13366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:33.899734 13366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:33.903740 13366 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:46695
I20260619 08:27:33.903777 13432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:46695 every 8 connection(s)
I20260619 08:27:33.904166 13366 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:33.905166 13433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:33.907510 13433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412: Bootstrap starting.
I20260619 08:27:33.908109 13433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:33.908437 13433 log.cc:824] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:33.908792 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 13366
I20260619 08:27:33.908923 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:33.909111 26110 external_mini_cluster.cc:1428] Setting key 9b762379fb60f2413c39856d75b8a48d
I20260619 08:27:33.909324 13433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412: No bootstrap required, opened a new log
I20260619 08:27:33.912282 13433 raft_consensus.cc:348] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce17afa27374293a037e52e1afd8412" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46695 } }
I20260619 08:27:33.912457 13433 raft_consensus.cc:374] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:33.912488 13433 raft_consensus.cc:729] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bce17afa27374293a037e52e1afd8412, State: Initialized, Role: FOLLOWER
I20260619 08:27:33.912606 13433 consensus_queue.cc:260] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce17afa27374293a037e52e1afd8412" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46695 } }
I20260619 08:27:33.912680 13433 raft_consensus.cc:388] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:33.912717 13433 raft_consensus.cc:482] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:33.912777 13433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:33.913349 13433 raft_consensus.cc:504] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce17afa27374293a037e52e1afd8412" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46695 } }
I20260619 08:27:33.913471 13433 leader_election.cc:302] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bce17afa27374293a037e52e1afd8412; no voters:
I20260619 08:27:33.913645 13433 leader_election.cc:288] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:33.913684 13438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:33.913828 13438 raft_consensus.cc:686] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412 [term 1 LEADER]: Becoming Leader. State: Replica: bce17afa27374293a037e52e1afd8412, State: Running, Role: LEADER
I20260619 08:27:33.913923 13433 sys_catalog.cc:564] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:33.913913 13438 consensus_queue.cc:237] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce17afa27374293a037e52e1afd8412" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46695 } }
I20260619 08:27:33.914314 13440 sys_catalog.cc:455] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bce17afa27374293a037e52e1afd8412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce17afa27374293a037e52e1afd8412" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46695 } } }
I20260619 08:27:33.914387 13440 sys_catalog.cc:458] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:33.914603 13439 sys_catalog.cc:455] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bce17afa27374293a037e52e1afd8412. Latest consensus state: current_term: 1 leader_uuid: "bce17afa27374293a037e52e1afd8412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce17afa27374293a037e52e1afd8412" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46695 } } }
I20260619 08:27:33.914674 13439 sys_catalog.cc:458] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:33.914675 13444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:33.915207 13444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:33.916376 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:33.909672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59642 (local address 127.25.127.190:46695)
0619 08:27:33.909899 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:33.909909 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:33.909948 (+ 39us) server_negotiation.cc:407] Connection header received
0619 08:27:33.910738 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:33.910748 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:33.910828 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:33.910986 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:33.911172 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.912658 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.913509 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:33.914212 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:33.914342 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:33.915141 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:33.915145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:33.915145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:33.915416 (+ 271us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:33.915467 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:33.915485 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:33.915986 (+ 501us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:33.916020 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 08:27:33.916101 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":87,"threads_started":1}
I20260619 08:27:33.916759 13444 catalog_manager.cc:1347] Generated new cluster ID: c2ba9aa902ee41b99de8eb6cb7c2d2bf
I20260619 08:27:33.916811 13444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:33.946422 13444 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:33.947173 13444 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:33.953778 13444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412: Generated new TSK 0
I20260619 08:27:33.954015 13444 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:33.962586 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.129:0
--local_ip_for_outbound_sockets=127.25.127.129
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:46695
--builtin_ntp_servers=127.25.127.148:42489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:27:34.082121 13457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:34.082432 13457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:34.082468 13457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:34.084285 13457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:34.084363 13457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:34.084405 13457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.129
I20260619 08:27:34.086575 13457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.127.129
--webserver_port=0
--tserver_master_addrs=127.25.127.190:46695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.129
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:34.086863 13457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:34.087222 13457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:34.092734 13462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:34.092762 13463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:34.092722 13465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:34.093194 13457 server_base.cc:1034] running on GCE node
I20260619 08:27:34.093425 13457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:34.093691 13457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:34.094863 13457 hybrid_clock.cc:648] HybridClock initialized: now 1781857654094839 us; error 38 us; skew 500 ppm
I20260619 08:27:34.096309 13457 webserver.cc:460] Webserver started at http://127.25.127.129:43607/ using document root <none> and password file <none>
I20260619 08:27:34.096592 13457 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:34.096668 13457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:34.096833 13457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:34.097944 13457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-0/data/instance:
uuid: "d65da295376d4f5b84b0f52578dcefd2"
format_stamp: "Formatted at 2026-06-19 08:27:34 on dist-test-slave-5rhg"
server_key: "d6d92d6c075f8ad89d9088b90bd94694"
server_key_iv: "9cbce7e9e3720765f8decf4850bddfb4"
server_key_version: "encryptionkey@0"
I20260619 08:27:34.098351 13457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance:
uuid: "d65da295376d4f5b84b0f52578dcefd2"
format_stamp: "Formatted at 2026-06-19 08:27:34 on dist-test-slave-5rhg"
server_key: "d6d92d6c075f8ad89d9088b90bd94694"
server_key_iv: "9cbce7e9e3720765f8decf4850bddfb4"
server_key_version: "encryptionkey@0"
I20260619 08:27:34.099694 13457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:27:34.100584 13471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:34.100766 13457 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:27:34.100873 13457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-0/wal
uuid: "d65da295376d4f5b84b0f52578dcefd2"
format_stamp: "Formatted at 2026-06-19 08:27:34 on dist-test-slave-5rhg"
server_key: "d6d92d6c075f8ad89d9088b90bd94694"
server_key_iv: "9cbce7e9e3720765f8decf4850bddfb4"
server_key_version: "encryptionkey@0"
I20260619 08:27:34.100955 13457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:34.119943 13457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:34.120409 13457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:34.120584 13457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:34.120836 13457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:34.121181 13457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:34.121234 13457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:34.121279 13457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:34.121304 13457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:34.127233 13457 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.129:33141
I20260619 08:27:34.127283 13584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.129:33141 every 8 connection(s)
I20260619 08:27:34.127622 13457 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-0/data/info.pb
I20260619 08:27:34.128911 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 13457
I20260619 08:27:34.128995 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-0/wal/instance
I20260619 08:27:34.129205 26110 external_mini_cluster.cc:1428] Setting key fcf307462d75a0f2b7baa29321f36cbe
I20260619 08:27:34.130334 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.130:0
--local_ip_for_outbound_sockets=127.25.127.130
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:46695
--builtin_ntp_servers=127.25.127.148:42489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 08:27:34.134091 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.128350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:33381 (local address 127.25.127.190:46695)
0619 08:27:34.128443 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:34.128445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:34.128698 (+ 253us) server_negotiation.cc:407] Connection header received
0619 08:27:34.129351 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:34.129352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:34.129370 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:34.129442 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:34.130520 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.131280 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.132456 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.132772 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.132860 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:34.133824 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:34.133825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:34.133826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:34.133830 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:34.133844 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:34.133848 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:34.133892 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:34.134010 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 08:27:34.134032 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 08:27:34.134377 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.128312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46695 (local address 127.25.127.129:33381)
0619 08:27:34.128554 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:34.128601 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:34.129200 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:34.129585 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:34.129590 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:34.129600 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:34.130388 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:34.130396 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.131376 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.131377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:34.132373 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:34.132376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.132874 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.132875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:34.133077 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:34.133668 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:27:34.133669 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:34.133708 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:27:34.133737 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:34.133912 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:34.133918 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:34.133921 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:27:34.133995 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 08:27:34.134122 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":108,"threads_started":1}
I20260619 08:27:34.134941 13585 heartbeater.cc:347] Connected to a master server at 127.25.127.190:46695
I20260619 08:27:34.135037 13585 heartbeater.cc:464] Registering TS with master...
I20260619 08:27:34.135238 13585 heartbeater.cc:511] Master 127.25.127.190:46695 requested a full tablet report, sending...
I20260619 08:27:34.135826 13397 ts_manager.cc:194] Registered new tserver with Master: d65da295376d4f5b84b0f52578dcefd2 (127.25.127.129:33141)
I20260619 08:27:34.136871 13397 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.127.129:33381
W20260619 08:27:34.214960 13588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:34.215260 13588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:34.215298 13588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:34.217464 13588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:34.217546 13588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:34.217595 13588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.130
I20260619 08:27:34.220152 13588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.127.130
--webserver_port=0
--tserver_master_addrs=127.25.127.190:46695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.130
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:34.220484 13588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:34.220876 13588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:34.225764 13596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:34.225809 13593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:34.225764 13594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:34.225991 13588 server_base.cc:1034] running on GCE node
I20260619 08:27:34.226197 13588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:34.226392 13588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:34.227556 13588 hybrid_clock.cc:648] HybridClock initialized: now 1781857654227518 us; error 62 us; skew 500 ppm
I20260619 08:27:34.228865 13588 webserver.cc:460] Webserver started at http://127.25.127.130:33791/ using document root <none> and password file <none>
I20260619 08:27:34.229131 13588 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:34.229204 13588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:34.229351 13588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:34.230439 13588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-1/data/instance:
uuid: "be664a1c5a91419890ff2aef7913fd95"
format_stamp: "Formatted at 2026-06-19 08:27:34 on dist-test-slave-5rhg"
server_key: "650d830a8d9f3375aefdf403fd54d0e6"
server_key_iv: "49f83c2a22353c0d6bc89bed69c131a6"
server_key_version: "encryptionkey@0"
I20260619 08:27:34.230816 13588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance:
uuid: "be664a1c5a91419890ff2aef7913fd95"
format_stamp: "Formatted at 2026-06-19 08:27:34 on dist-test-slave-5rhg"
server_key: "650d830a8d9f3375aefdf403fd54d0e6"
server_key_iv: "49f83c2a22353c0d6bc89bed69c131a6"
server_key_version: "encryptionkey@0"
I20260619 08:27:34.232127 13588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:27:34.232918 13602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:34.233098 13588 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:27:34.233196 13588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-1/wal
uuid: "be664a1c5a91419890ff2aef7913fd95"
format_stamp: "Formatted at 2026-06-19 08:27:34 on dist-test-slave-5rhg"
server_key: "650d830a8d9f3375aefdf403fd54d0e6"
server_key_iv: "49f83c2a22353c0d6bc89bed69c131a6"
server_key_version: "encryptionkey@0"
I20260619 08:27:34.233282 13588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:34.247916 13588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:34.248310 13588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:34.248471 13588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:34.248708 13588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:34.249054 13588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:34.249109 13588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:34.249152 13588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:34.249194 13588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:34.255280 13588 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.130:34311
I20260619 08:27:34.255326 13715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.130:34311 every 8 connection(s)
I20260619 08:27:34.255681 13588 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-1/data/info.pb
I20260619 08:27:34.256043 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 13588
I20260619 08:27:34.256182 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-1/wal/instance
I20260619 08:27:34.256424 26110 external_mini_cluster.cc:1428] Setting key 4f27a920a7b5195f84d7de29d77efacc
I20260619 08:27:34.257462 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.127.131:0
--local_ip_for_outbound_sockets=127.25.127.131
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:46695
--builtin_ntp_servers=127.25.127.148:42489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 08:27:34.261900 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.256502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.130:39065 (local address 127.25.127.190:46695)
0619 08:27:34.256553 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:34.256556 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:34.256855 (+ 299us) server_negotiation.cc:407] Connection header received
0619 08:27:34.257418 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:34.257419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:34.257432 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:34.257506 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:34.258286 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.258984 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.260309 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.260645 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.260721 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:34.261594 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:34.261596 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:34.261596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:34.261603 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:34.261621 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:34.261625 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:34.261697 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:34.261748 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 08:27:34.261777 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:27:34.262038 13718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.256481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46695 (local address 127.25.127.130:39065)
0619 08:27:34.256685 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:34.256731 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:34.257317 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:34.257613 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:34.257619 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:34.257631 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:34.258191 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:34.258197 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.259086 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.259087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:34.260230 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:34.260234 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.260733 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.260733 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:34.260848 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:34.261440 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:27:34.261440 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:34.261471 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:27:34.261494 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:34.261695 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:34.261697 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:34.261700 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:27:34.261746 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 08:27:34.261829 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260619 08:27:34.262609 13716 heartbeater.cc:347] Connected to a master server at 127.25.127.190:46695
I20260619 08:27:34.262719 13716 heartbeater.cc:464] Registering TS with master...
I20260619 08:27:34.262923 13716 heartbeater.cc:511] Master 127.25.127.190:46695 requested a full tablet report, sending...
I20260619 08:27:34.263324 13397 ts_manager.cc:194] Registered new tserver with Master: be664a1c5a91419890ff2aef7913fd95 (127.25.127.130:34311)
I20260619 08:27:34.263707 13397 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.127.130:39065
W20260619 08:27:34.339709 13719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:34.339913 13719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:34.339937 13719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:34.341157 13719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:34.341199 13719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:34.341226 13719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.127.131
I20260619 08:27:34.342650 13719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:42489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.127.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.127.131
--webserver_port=0
--tserver_master_addrs=127.25.127.190:46695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.127.131
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:34.342885 13719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:34.343183 13719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:34.348351 13724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:34.348381 13725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:34.348420 13719 server_base.cc:1034] running on GCE node
W20260619 08:27:34.348385 13727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:34.348774 13719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:34.349032 13719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:34.350200 13719 hybrid_clock.cc:648] HybridClock initialized: now 1781857654350194 us; error 36 us; skew 500 ppm
I20260619 08:27:34.351579 13719 webserver.cc:460] Webserver started at http://127.25.127.131:34991/ using document root <none> and password file <none>
I20260619 08:27:34.351830 13719 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:34.351902 13719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:34.352063 13719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:34.353175 13719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-2/data/instance:
uuid: "81f8468100f24ed2b8d83bd1b118cff1"
format_stamp: "Formatted at 2026-06-19 08:27:34 on dist-test-slave-5rhg"
server_key: "dec7256b3424e2c2e00c61020fe5fb75"
server_key_iv: "33ed7f31c7cd99617660697542c35ce7"
server_key_version: "encryptionkey@0"
I20260619 08:27:34.353557 13719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance:
uuid: "81f8468100f24ed2b8d83bd1b118cff1"
format_stamp: "Formatted at 2026-06-19 08:27:34 on dist-test-slave-5rhg"
server_key: "dec7256b3424e2c2e00c61020fe5fb75"
server_key_iv: "33ed7f31c7cd99617660697542c35ce7"
server_key_version: "encryptionkey@0"
I20260619 08:27:34.354890 13719 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:27:34.355635 13733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:34.355813 13719 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:27:34.355912 13719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-2/wal
uuid: "81f8468100f24ed2b8d83bd1b118cff1"
format_stamp: "Formatted at 2026-06-19 08:27:34 on dist-test-slave-5rhg"
server_key: "dec7256b3424e2c2e00c61020fe5fb75"
server_key_iv: "33ed7f31c7cd99617660697542c35ce7"
server_key_version: "encryptionkey@0"
I20260619 08:27:34.356001 13719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:34.366703 13719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:34.367053 13719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:34.367209 13719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:34.367441 13719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:27:34.367805 13719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:27:34.367861 13719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:34.367904 13719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:27:34.367933 13719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:34.373693 13719 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.131:36585
I20260619 08:27:34.373798 13846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.131:36585 every 8 connection(s)
I20260619 08:27:34.374135 13719 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-2/data/info.pb
I20260619 08:27:34.379455 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.374933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.131:60567 (local address 127.25.127.190:46695)
0619 08:27:34.375006 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:34.375008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:34.375266 (+ 258us) server_negotiation.cc:407] Connection header received
0619 08:27:34.375806 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:34.375807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:34.375818 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:34.375872 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:34.376602 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.377110 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.378109 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.378327 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.378405 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:34.379184 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:34.379186 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:34.379186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:34.379191 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:34.379206 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:34.379210 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:34.379253 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:34.379351 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 08:27:34.379377 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 08:27:34.379707 13849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.374919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46695 (local address 127.25.127.131:60567)
0619 08:27:34.375130 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:34.375166 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:34.375692 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:34.375959 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:34.375963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:34.375971 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:34.376494 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:34.376498 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.377211 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.377212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:34.377998 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:34.378000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.378413 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.378414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:34.378545 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:34.379034 (+ 489us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:27:34.379035 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:34.379075 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:27:34.379096 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:34.379264 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:34.379269 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:34.379271 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:27:34.379354 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 08:27:34.379458 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":89,"threads_started":1}
I20260619 08:27:34.380302 13847 heartbeater.cc:347] Connected to a master server at 127.25.127.190:46695
I20260619 08:27:34.380403 13847 heartbeater.cc:464] Registering TS with master...
I20260619 08:27:34.380591 13847 heartbeater.cc:511] Master 127.25.127.190:46695 requested a full tablet report, sending...
I20260619 08:27:34.380990 13397 ts_manager.cc:194] Registered new tserver with Master: 81f8468100f24ed2b8d83bd1b118cff1 (127.25.127.131:36585)
I20260619 08:27:34.381351 13397 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.127.131:60567
I20260619 08:27:34.382933 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 13719
I20260619 08:27:34.383056 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-2/wal/instance
I20260619 08:27:34.383280 26110 external_mini_cluster.cc:1428] Setting key f4ed0f411e0ec8e8ca264b2825cfd15f
I20260619 08:27:34.384178 26110 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:27:34.388361 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.385586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59644 (local address 127.25.127.190:46695)
0619 08:27:34.385624 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:34.385626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:34.385718 (+ 92us) server_negotiation.cc:407] Connection header received
0619 08:27:34.385757 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:34.385758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:34.385766 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:34.385849 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:34.385972 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.386431 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.387151 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.387345 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.387423 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:34.388088 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:34.388090 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:34.388090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:34.388094 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:34.388108 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:34.388112 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:34.388238 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:34.388282 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 08:27:34.388303 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 08:27:34.389894 13397 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:27:34.391434 13397 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:27:34.398214 13857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.393801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36604 (local address 127.25.127.131:36585)
0619 08:27:34.394344 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:34.394347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:34.394356 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:27:34.394393 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:34.394395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:34.394409 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:34.394481 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:34.394663 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.395834 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.397059 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.397801 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.397884 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:34.397983 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:34.398078 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 08:27:34.398123 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":96,"threads_started":1}
I20260619 08:27:34.398208 13858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.393736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:33141 (local address 127.0.0.1:51114)
0619 08:27:34.394072 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:34.394084 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:34.394136 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:34.394472 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:34.394475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:34.394477 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:34.394597 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:34.394599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.395628 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.395629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:34.396915 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:34.396918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.397896 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.397897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:34.397982 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:34.397988 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 08:27:34.398028 (+ 40us) client_negotiation.cc:240] Negotiation successful
0619 08:27:34.398121 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":2,"thread_start_us":52,"threads_started":1}
I20260619 08:27:34.398216 13856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.393696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:36585 (local address 127.0.0.1:36604)
0619 08:27:34.393872 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:34.393922 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:34.393985 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:34.394546 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:34.394547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:34.394550 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:34.394593 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:34.394595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.395921 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.395922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:34.396982 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:34.396984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.397880 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.397880 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:34.397961 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:34.397977 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 08:27:34.398049 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 08:27:34.398127 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":29,"thread_start_us":73,"threads_started":1}
I20260619 08:27:34.398983 13855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.393930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:34311 (local address 127.0.0.1:36588)
0619 08:27:34.394104 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:34.394114 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:34.394152 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:34.394509 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:34.394510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:34.394512 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:34.394622 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:34.394624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.397073 (+ 2449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.397074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:34.397735 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:34.397736 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.398765 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.398766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:34.398828 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:34.398832 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:27:34.398876 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 08:27:34.398917 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":52,"thread_start_us":99,"threads_started":1}
I20260619 08:27:34.399015 13859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.394038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36588 (local address 127.25.127.130:34311)
0619 08:27:34.394246 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:34.394251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:34.394268 (+ 17us) server_negotiation.cc:407] Connection header received
0619 08:27:34.394343 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:34.394345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:34.394362 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:34.394436 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:34.394706 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.395834 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.397823 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.398695 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.398767 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:34.398853 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:34.398884 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 08:27:34.398923 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":96,"threads_started":1}
I20260619 08:27:34.399127 13781 tablet_service.cc:1467] Processing CreateTablet for tablet 5b78541429ff46d4a4fd4e946266b0be (DEFAULT_TABLE table=test-table [id=69f13f395bc840e59ad74919443bcafa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:27:34.399533 13781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b78541429ff46d4a4fd4e946266b0be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:34.399765 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 5b78541429ff46d4a4fd4e946266b0be (DEFAULT_TABLE table=test-table [id=69f13f395bc840e59ad74919443bcafa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:27:34.400091 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b78541429ff46d4a4fd4e946266b0be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:34.403302 13861 tablet_bootstrap.cc:492] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1: Bootstrap starting.
I20260619 08:27:34.403561 13860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.394109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51114 (local address 127.25.127.129:33141)
0619 08:27:34.394274 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:34.394279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:34.394289 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:27:34.394332 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:34.394334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:34.394350 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:34.394681 (+ 331us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:34.394770 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.395539 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.397010 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.397802 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.403236 (+ 5434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:34.403392 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:34.403425 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 08:27:34.403479 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260619 08:27:34.403556 13862 tablet_bootstrap.cc:492] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95: Bootstrap starting.
I20260619 08:27:34.404032 13861 tablet_bootstrap.cc:654] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:34.404376 13861 log.cc:824] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:34.404294 13519 tablet_service.cc:1467] Processing CreateTablet for tablet 5b78541429ff46d4a4fd4e946266b0be (DEFAULT_TABLE table=test-table [id=69f13f395bc840e59ad74919443bcafa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:27:34.404448 13862 tablet_bootstrap.cc:654] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:34.404654 13519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b78541429ff46d4a4fd4e946266b0be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:34.404755 13862 log.cc:824] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:34.405251 13861 tablet_bootstrap.cc:492] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1: No bootstrap required, opened a new log
I20260619 08:27:34.405337 13861 ts_tablet_manager.cc:1397] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260619 08:27:34.405496 13862 tablet_bootstrap.cc:492] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95: No bootstrap required, opened a new log
I20260619 08:27:34.405568 13862 ts_tablet_manager.cc:1397] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:27:34.407840 13861 raft_consensus.cc:348] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81f8468100f24ed2b8d83bd1b118cff1" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 36585 } } peers { permanent_uuid: "be664a1c5a91419890ff2aef7913fd95" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34311 } } peers { permanent_uuid: "d65da295376d4f5b84b0f52578dcefd2" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33141 } }
I20260619 08:27:34.407998 13861 raft_consensus.cc:374] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:34.408042 13861 raft_consensus.cc:729] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81f8468100f24ed2b8d83bd1b118cff1, State: Initialized, Role: FOLLOWER
I20260619 08:27:34.408197 13861 consensus_queue.cc:260] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81f8468100f24ed2b8d83bd1b118cff1" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 36585 } } peers { permanent_uuid: "be664a1c5a91419890ff2aef7913fd95" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34311 } } peers { permanent_uuid: "d65da295376d4f5b84b0f52578dcefd2" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33141 } }
I20260619 08:27:34.408293 13865 tablet_bootstrap.cc:492] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2: Bootstrap starting.
I20260619 08:27:34.408318 13862 raft_consensus.cc:348] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81f8468100f24ed2b8d83bd1b118cff1" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 36585 } } peers { permanent_uuid: "be664a1c5a91419890ff2aef7913fd95" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34311 } } peers { permanent_uuid: "d65da295376d4f5b84b0f52578dcefd2" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33141 } }
I20260619 08:27:34.408478 13862 raft_consensus.cc:374] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:34.408499 13861 ts_tablet_manager.cc:1428] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260619 08:27:34.408514 13862 raft_consensus.cc:729] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be664a1c5a91419890ff2aef7913fd95, State: Initialized, Role: FOLLOWER
I20260619 08:27:34.408638 13847 heartbeater.cc:503] Master 127.25.127.190:46695 was elected leader, sending a full tablet report...
I20260619 08:27:34.408665 13862 consensus_queue.cc:260] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81f8468100f24ed2b8d83bd1b118cff1" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 36585 } } peers { permanent_uuid: "be664a1c5a91419890ff2aef7913fd95" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34311 } } peers { permanent_uuid: "d65da295376d4f5b84b0f52578dcefd2" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33141 } }
I20260619 08:27:34.408923 13862 ts_tablet_manager.cc:1428] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 08:27:34.408962 13865 tablet_bootstrap.cc:654] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:34.408959 13716 heartbeater.cc:503] Master 127.25.127.190:46695 was elected leader, sending a full tablet report...
I20260619 08:27:34.409302 13865 log.cc:824] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:34.410207 13865 tablet_bootstrap.cc:492] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2: No bootstrap required, opened a new log
I20260619 08:27:34.410260 13865 ts_tablet_manager.cc:1397] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:27:34.412019 13865 raft_consensus.cc:348] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81f8468100f24ed2b8d83bd1b118cff1" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 36585 } } peers { permanent_uuid: "be664a1c5a91419890ff2aef7913fd95" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34311 } } peers { permanent_uuid: "d65da295376d4f5b84b0f52578dcefd2" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33141 } }
I20260619 08:27:34.412194 13865 raft_consensus.cc:374] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:34.412243 13865 raft_consensus.cc:729] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d65da295376d4f5b84b0f52578dcefd2, State: Initialized, Role: FOLLOWER
I20260619 08:27:34.412354 13865 consensus_queue.cc:260] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81f8468100f24ed2b8d83bd1b118cff1" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 36585 } } peers { permanent_uuid: "be664a1c5a91419890ff2aef7913fd95" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34311 } } peers { permanent_uuid: "d65da295376d4f5b84b0f52578dcefd2" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33141 } }
I20260619 08:27:34.412604 13865 ts_tablet_manager.cc:1428] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:27:34.412665 13585 heartbeater.cc:503] Master 127.25.127.190:46695 was elected leader, sending a full tablet report...
I20260619 08:27:34.489728 13869 raft_consensus.cc:482] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:27:34.489897 13869 raft_consensus.cc:504] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81f8468100f24ed2b8d83bd1b118cff1" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 36585 } } peers { permanent_uuid: "be664a1c5a91419890ff2aef7913fd95" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34311 } } peers { permanent_uuid: "d65da295376d4f5b84b0f52578dcefd2" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33141 } }
I20260619 08:27:34.490330 13869 leader_election.cc:288] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81f8468100f24ed2b8d83bd1b118cff1 (127.25.127.131:36585), be664a1c5a91419890ff2aef7913fd95 (127.25.127.130:34311)
I20260619 08:27:34.493320 13857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.490544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:53769 (local address 127.25.127.131:36585)
0619 08:27:34.490603 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:34.490607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:34.490622 (+ 15us) server_negotiation.cc:407] Connection header received
0619 08:27:34.490650 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:34.490650 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:34.490658 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:34.490744 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:34.490928 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.491448 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.492552 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.492929 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.493024 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:34.493033 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:34.493197 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 08:27:34.493239 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 08:27:34.493350 13859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.490751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.127.129:45785 (local address 127.25.127.130:34311)
0619 08:27:34.490803 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:34.490805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:34.490839 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:27:34.490907 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:34.490908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:34.490916 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:34.490967 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:27:34.491128 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.491599 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.492606 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.492978 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.493079 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:34.493087 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:34.493235 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 08:27:34.493275 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:27:34.493366 13870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.490601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.130:34311 (local address 127.25.127.129:45785)
0619 08:27:34.490782 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:34.490792 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:34.490843 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:34.490972 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:34.490972 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:34.490974 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:34.491048 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:34.491049 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.491685 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.491686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:34.492509 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:34.492512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.493084 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.493086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:34.493160 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:34.493163 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:27:34.493230 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 08:27:34.493271 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":104,"threads_started":1}
I20260619 08:27:34.493353 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.490475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.131:36585 (local address 127.25.127.129:53769)
0619 08:27:34.490542 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:34.490555 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:34.490599 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:34.490766 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:34.490768 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:34.490770 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:27:34.490847 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:34.490849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.491540 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.491541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:34.492439 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:34.492443 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.493026 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.493027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:34.493108 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:34.493112 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:27:34.493188 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 08:27:34.493235 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":24}
I20260619 08:27:34.493723 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b78541429ff46d4a4fd4e946266b0be" candidate_uuid: "d65da295376d4f5b84b0f52578dcefd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be664a1c5a91419890ff2aef7913fd95" is_pre_election: true
I20260619 08:27:34.493878 13801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b78541429ff46d4a4fd4e946266b0be" candidate_uuid: "d65da295376d4f5b84b0f52578dcefd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81f8468100f24ed2b8d83bd1b118cff1" is_pre_election: true
I20260619 08:27:34.493922 13670 raft_consensus.cc:2454] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d65da295376d4f5b84b0f52578dcefd2 in term 0.
I20260619 08:27:34.494017 13801 raft_consensus.cc:2454] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d65da295376d4f5b84b0f52578dcefd2 in term 0.
I20260619 08:27:34.494176 13473 leader_election.cc:302] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be664a1c5a91419890ff2aef7913fd95, d65da295376d4f5b84b0f52578dcefd2; no voters:
I20260619 08:27:34.494355 13869 raft_consensus.cc:2781] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:27:34.494442 13869 raft_consensus.cc:482] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:27:34.494480 13869 raft_consensus.cc:3037] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:34.495337 13869 raft_consensus.cc:504] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81f8468100f24ed2b8d83bd1b118cff1" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 36585 } } peers { permanent_uuid: "be664a1c5a91419890ff2aef7913fd95" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34311 } } peers { permanent_uuid: "d65da295376d4f5b84b0f52578dcefd2" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33141 } }
I20260619 08:27:34.495486 13869 leader_election.cc:288] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [CANDIDATE]: Term 1 election: Requested vote from peers 81f8468100f24ed2b8d83bd1b118cff1 (127.25.127.131:36585), be664a1c5a91419890ff2aef7913fd95 (127.25.127.130:34311)
I20260619 08:27:34.495659 13801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b78541429ff46d4a4fd4e946266b0be" candidate_uuid: "d65da295376d4f5b84b0f52578dcefd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81f8468100f24ed2b8d83bd1b118cff1"
I20260619 08:27:34.495702 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b78541429ff46d4a4fd4e946266b0be" candidate_uuid: "d65da295376d4f5b84b0f52578dcefd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be664a1c5a91419890ff2aef7913fd95"
I20260619 08:27:34.495751 13801 raft_consensus.cc:3037] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:34.495774 13670 raft_consensus.cc:3037] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:34.496709 13801 raft_consensus.cc:2454] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d65da295376d4f5b84b0f52578dcefd2 in term 1.
I20260619 08:27:34.496709 13670 raft_consensus.cc:2454] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d65da295376d4f5b84b0f52578dcefd2 in term 1.
I20260619 08:27:34.496882 13473 leader_election.cc:302] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be664a1c5a91419890ff2aef7913fd95, d65da295376d4f5b84b0f52578dcefd2; no voters:
I20260619 08:27:34.496968 13869 raft_consensus.cc:2781] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:34.497143 13869 raft_consensus.cc:686] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [term 1 LEADER]: Becoming Leader. State: Replica: d65da295376d4f5b84b0f52578dcefd2, State: Running, Role: LEADER
I20260619 08:27:34.497248 13869 consensus_queue.cc:237] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81f8468100f24ed2b8d83bd1b118cff1" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 36585 } } peers { permanent_uuid: "be664a1c5a91419890ff2aef7913fd95" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34311 } } peers { permanent_uuid: "d65da295376d4f5b84b0f52578dcefd2" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33141 } }
I20260619 08:27:34.498172 13396 catalog_manager.cc:5495] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to d65da295376d4f5b84b0f52578dcefd2 (127.25.127.129). New cstate: current_term: 1 leader_uuid: "d65da295376d4f5b84b0f52578dcefd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81f8468100f24ed2b8d83bd1b118cff1" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 36585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be664a1c5a91419890ff2aef7913fd95" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d65da295376d4f5b84b0f52578dcefd2" member_type: VOTER last_known_addr { host: "127.25.127.129" port: 33141 } health_report { overall_health: HEALTHY } } }
W20260619 08:27:34.506222 13717 tablet.cc:2369] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:27:34.517899 13860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.515537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51118 (local address 127.25.127.129:33141)
0619 08:27:34.515599 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:34.515601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:34.515610 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:27:34.515656 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:34.515657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:34.515681 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:34.515754 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:27:34.515887 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.516489 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.517225 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:34.517403 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:34.517452 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:34.517580 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:27:34.517681 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:27:34.517741 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:34.517760 (+ 19us) server_negotiation.cc:299] Negotiation successful
0619 08:27:34.517775 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 08:27:34.519778 13670 raft_consensus.cc:1261] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95 [term 1 FOLLOWER]: Refusing update from remote peer d65da295376d4f5b84b0f52578dcefd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:27:34.519778 13801 raft_consensus.cc:1261] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1 [term 1 FOLLOWER]: Refusing update from remote peer d65da295376d4f5b84b0f52578dcefd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:27:34.520181 13871 consensus_queue.cc:1035] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81f8468100f24ed2b8d83bd1b118cff1" member_type: VOTER last_known_addr { host: "127.25.127.131" port: 36585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:27:34.520316 13869 consensus_queue.cc:1035] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be664a1c5a91419890ff2aef7913fd95" member_type: VOTER last_known_addr { host: "127.25.127.130" port: 34311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:27:34.523270 13876 mvcc.cc:205] Tried to move back new op lower bound from 7298488952910000128 to 7298488952821575680. Current Snapshot: MvccSnapshot[applied={T|T < 7298488952910000128}]
I20260619 08:27:34.523670 13875 mvcc.cc:205] Tried to move back new op lower bound from 7298488952910000128 to 7298488952821575680. Current Snapshot: MvccSnapshot[applied={T|T < 7298488952910000128}]
I20260619 08:27:34.525141 13397 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:27:34.525242 13397 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:27:34.526024 13397 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bce17afa27374293a037e52e1afd8412: Sending DeleteTablet for 3 replicas of tablet 5b78541429ff46d4a4fd4e946266b0be
I20260619 08:27:34.526355 13519 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b78541429ff46d4a4fd4e946266b0be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:27:34 UTC) from {username='slave'} at 127.0.0.1:51114
I20260619 08:27:34.526474 13781 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b78541429ff46d4a4fd4e946266b0be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:27:34 UTC) from {username='slave'} at 127.0.0.1:36604
I20260619 08:27:34.526605 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 13457
I20260619 08:27:34.526739 13884 tablet_replica.cc:330] T 5b78541429ff46d4a4fd4e946266b0be P d65da295376d4f5b84b0f52578dcefd2: stopping tablet replica
I20260619 08:27:34.526717 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b78541429ff46d4a4fd4e946266b0be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:27:34 UTC) from {username='slave'} at 127.0.0.1:36588
I20260619 08:27:34.526926 13885 tablet_replica.cc:330] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1: stopping tablet replica
I20260619 08:27:34.527238 13885 raft_consensus.cc:2229] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:27:34.527334 13885 raft_consensus.cc:2258] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:27:34.527707 13885 ts_tablet_manager.cc:1905] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:27:34.527892 13886 tablet_replica.cc:330] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95: stopping tablet replica
I20260619 08:27:34.528002 13886 raft_consensus.cc:2229] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:27:34.528106 13886 raft_consensus.cc:2258] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:27:34.528525 13886 ts_tablet_manager.cc:1905] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:27:34.529610 13885 ts_tablet_manager.cc:1918] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:27:34.529711 13885 log.cc:1196] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-2/wal/wals/5b78541429ff46d4a4fd4e946266b0be
I20260619 08:27:34.529942 13885 ts_tablet_manager.cc:1939] T 5b78541429ff46d4a4fd4e946266b0be P 81f8468100f24ed2b8d83bd1b118cff1: Deleting consensus metadata
I20260619 08:27:34.530366 13382 catalog_manager.cc:4841] TS 81f8468100f24ed2b8d83bd1b118cff1 (127.25.127.131:36585): tablet 5b78541429ff46d4a4fd4e946266b0be (table test-table [id=69f13f395bc840e59ad74919443bcafa]) successfully deleted
I20260619 08:27:34.530413 13886 ts_tablet_manager.cc:1918] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:27:34.530478 13886 log.cc:1196] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95: Deleting WAL directory at /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781857412505079-26110-0/minicluster-data/ts-1/wal/wals/5b78541429ff46d4a4fd4e946266b0be
I20260619 08:27:34.530727 13886 ts_tablet_manager.cc:1939] T 5b78541429ff46d4a4fd4e946266b0be P be664a1c5a91419890ff2aef7913fd95: Deleting consensus metadata
I20260619 08:27:34.531114 13382 catalog_manager.cc:4841] TS be664a1c5a91419890ff2aef7913fd95 (127.25.127.130:34311): tablet 5b78541429ff46d4a4fd4e946266b0be (table test-table [id=69f13f395bc840e59ad74919443bcafa]) successfully deleted
W20260619 08:27:34.533854 13384 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.127.129:33141 (error 108)
I20260619 08:27:34.534098 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 13588
I20260619 08:27:34.534911 13855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:34.534761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.129:33141 (local address 127.0.0.1:51126)
0619 08:27:34.534821 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:34.534869 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.129:33141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 08:27:34.535055 13384 catalog_manager.cc:4584] TS d65da295376d4f5b84b0f52578dcefd2 (127.25.127.129:33141): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b78541429ff46d4a4fd4e946266b0be: Network error: Client connection negotiation failed: client connection to 127.25.127.129:33141: connect: Connection refused (error 111)
I20260619 08:27:34.539258 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 13719
I20260619 08:27:34.543807 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 13366
2026-06-19T08:27:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (786 ms)
[----------] 6 tests from EncryptionPolicyTest (8577 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:34 dist-test-slave-5rhg krb5kdc[13890](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:34 dist-test-slave-5rhg krb5kdc[13890](info): set up 2 sockets
Jun 19 08:27:34 dist-test-slave-5rhg krb5kdc[13890](info): commencing operation
krb5kdc: starting...
W20260619 08:27:35.601065 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:35 dist-test-slave-5rhg krb5kdc[13890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:27:35.718860 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37823
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:35.801308 13903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:35.801522 13903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:35.801546 13903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:35.802677 13903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:35.802719 13903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:35.802733 13903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:35.802764 13903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:35.802776 13903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:35.804669 13903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37823
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:35.804883 13903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:35.805058 13903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:35.809999 13911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:35.809975 13909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:35.810248 13903 server_base.cc:1034] running on GCE node
W20260619 08:27:35.809958 13908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:35.810530 13903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:35.810786 13903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:35.811990 13903 hybrid_clock.cc:648] HybridClock initialized: now 1781857655811975 us; error 29 us; skew 500 ppm
Jun 19 08:27:35 dist-test-slave-5rhg krb5kdc[13890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857655, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:35.814606 13903 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:27:35.815035 13903 webserver.cc:460] Webserver started at http://127.0.0.1:43013/ using document root <none> and password file <none>
I20260619 08:27:35.815279 13903 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:35.815351 13903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:35.815505 13903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:35.816689 13903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "97b7bd1361c4427eb1d1d01ab2aece34"
format_stamp: "Formatted at 2026-06-19 08:27:35 on dist-test-slave-5rhg"
server_key: "9898ec4d2cbed6fc122b0bb27f10c80d"
server_key_iv: "5d916412011ca8c9ecc1e30ad718569c"
server_key_version: "encryptionkey@0"
I20260619 08:27:35.817063 13903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "97b7bd1361c4427eb1d1d01ab2aece34"
format_stamp: "Formatted at 2026-06-19 08:27:35 on dist-test-slave-5rhg"
server_key: "9898ec4d2cbed6fc122b0bb27f10c80d"
server_key_iv: "5d916412011ca8c9ecc1e30ad718569c"
server_key_version: "encryptionkey@0"
I20260619 08:27:35.818408 13903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 08:27:35.819144 13918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:35.819319 13903 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:27:35.819427 13903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "97b7bd1361c4427eb1d1d01ab2aece34"
format_stamp: "Formatted at 2026-06-19 08:27:35 on dist-test-slave-5rhg"
server_key: "9898ec4d2cbed6fc122b0bb27f10c80d"
server_key_iv: "5d916412011ca8c9ecc1e30ad718569c"
server_key_version: "encryptionkey@0"
I20260619 08:27:35.819504 13903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:35.841477 13903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:35.843959 13903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:35.844130 13903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:35.847981 13903 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37823
I20260619 08:27:35.848042 13970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37823 every 8 connection(s)
I20260619 08:27:35.848444 13903 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:35.849316 13971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:35.851689 13971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34: Bootstrap starting.
I20260619 08:27:35.852283 13971 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:35.852579 13971 log.cc:824] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:35.853302 13971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34: No bootstrap required, opened a new log
I20260619 08:27:35.853922 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 13903
I20260619 08:27:35.854036 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:35.854231 26110 external_mini_cluster.cc:1428] Setting key b2b2c6670694fcd638012198553ae227
I20260619 08:27:35.855578 13971 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97b7bd1361c4427eb1d1d01ab2aece34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37823 } }
I20260619 08:27:35.855713 13971 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:35.855736 13971 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b7bd1361c4427eb1d1d01ab2aece34, State: Initialized, Role: FOLLOWER
I20260619 08:27:35.855862 13971 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97b7bd1361c4427eb1d1d01ab2aece34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37823 } }
I20260619 08:27:35.855930 13971 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:35.855957 13971 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:35.856006 13971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:35.856593 13971 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97b7bd1361c4427eb1d1d01ab2aece34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37823 } }
I20260619 08:27:35.856710 13971 leader_election.cc:302] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97b7bd1361c4427eb1d1d01ab2aece34; no voters:
I20260619 08:27:35.856911 13971 leader_election.cc:288] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:35.856940 13976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:35.857053 13976 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34 [term 1 LEADER]: Becoming Leader. State: Replica: 97b7bd1361c4427eb1d1d01ab2aece34, State: Running, Role: LEADER
I20260619 08:27:35.857138 13976 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97b7bd1361c4427eb1d1d01ab2aece34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37823 } }
I20260619 08:27:35.857183 13971 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:35.857434 13976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97b7bd1361c4427eb1d1d01ab2aece34. Latest consensus state: current_term: 1 leader_uuid: "97b7bd1361c4427eb1d1d01ab2aece34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97b7bd1361c4427eb1d1d01ab2aece34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37823 } } }
I20260619 08:27:35.857508 13976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:35.857445 13977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97b7bd1361c4427eb1d1d01ab2aece34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97b7bd1361c4427eb1d1d01ab2aece34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37823 } } }
I20260619 08:27:35.857812 13977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34 [sys.catalog]: This master's current role is: LEADER
W20260619 08:27:35.859140 13991 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:27:35.859202 13991 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:27:35.859319 13992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:35.859495 13992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:35.861063 13992 catalog_manager.cc:1347] Generated new cluster ID: 702d57dfa6124ad0a64e0dcd2b13236d
I20260619 08:27:35.861119 13992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:27:35 dist-test-slave-5rhg krb5kdc[13890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 08:27:35.865998 13975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:35.854800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52705 (local address 127.0.0.1:37823)
0619 08:27:35.855003 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:35.855011 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:35.855036 (+ 25us) server_negotiation.cc:407] Connection header received
0619 08:27:35.855766 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:35.855777 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:35.855864 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:35.856046 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:35.856762 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:35.858188 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:35.859248 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:35.859875 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:35.859985 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:35.862603 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:35.862607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:35.862607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:35.862619 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:35.864441 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:35.864836 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:35.864839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:35.864839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:35.864937 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:35.865168 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:35.865169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:35.865170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:35.865401 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:35.865575 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:35.865716 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 08:27:35.865821 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260619 08:27:35.867024 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:27:35.876638 13992 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:35.877358 13992 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:35.878607 13975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:35.868391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60429 (local address 127.0.0.1:37823)
0619 08:27:35.868433 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:35.868435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:35.872340 (+ 3905us) server_negotiation.cc:407] Connection header received
0619 08:27:35.872417 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:35.872418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:35.872428 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:35.872483 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:35.873002 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:35.873516 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:35.874271 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:35.874508 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:35.874603 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:35.875826 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:35.875827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:35.875827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:35.875834 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:35.877368 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:35.877789 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:35.877790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:35.877790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:35.877849 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:35.878072 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:35.878073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:35.878074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:35.878185 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:35.878296 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:35.878418 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 08:27:35.878480 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:27:35.887287 13992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97b7bd1361c4427eb1d1d01ab2aece34: Generated new TSK 0
I20260619 08:27:35.887504 13992 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:35.891242 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 13903
W20260619 08:27:35.895768 13888 connection.cc:535] client connection to 127.0.0.1:37823 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37823): Connection reset by peer (error 104)
2026-06-19T08:27:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1345 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:35 dist-test-slave-5rhg krb5kdc[14002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:35 dist-test-slave-5rhg krb5kdc[14002](info): set up 2 sockets
Jun 19 08:27:35 dist-test-slave-5rhg krb5kdc[14002](info): commencing operation
krb5kdc: starting...
W20260619 08:27:36.929049 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:37 dist-test-slave-5rhg krb5kdc[14002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:27:37.044595 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44995
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:37.126071 14015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:37.126328 14015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:37.126366 14015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:37.127562 14015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:37.127636 14015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:37.127663 14015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:37.127694 14015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:37.127719 14015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:37.129274 14015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44995
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:37.129573 14015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:37.129832 14015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:37.134784 14020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:37.134971 14015 server_base.cc:1034] running on GCE node
W20260619 08:27:37.134820 14023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:37.134784 14021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:37.135321 14015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:37.135594 14015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:37.136730 14015 hybrid_clock.cc:648] HybridClock initialized: now 1781857657136717 us; error 28 us; skew 500 ppm
Jun 19 08:27:37 dist-test-slave-5rhg krb5kdc[14002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857657, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:37.142545 14015 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:27:37.142987 14015 webserver.cc:460] Webserver started at http://127.0.0.1:37969/ using document root <none> and password file <none>
I20260619 08:27:37.143227 14015 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:37.143275 14015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:37.143419 14015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:37.144600 14015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "b466d5f9a39248cd94d770768c0c594f"
format_stamp: "Formatted at 2026-06-19 08:27:37 on dist-test-slave-5rhg"
server_key: "ad3e68a46806667b8dff05efb39ac548"
server_key_iv: "6fcbf34b1bbbfd6436d140d8aacd84ec"
server_key_version: "encryptionkey@0"
I20260619 08:27:37.144951 14015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "b466d5f9a39248cd94d770768c0c594f"
format_stamp: "Formatted at 2026-06-19 08:27:37 on dist-test-slave-5rhg"
server_key: "ad3e68a46806667b8dff05efb39ac548"
server_key_iv: "6fcbf34b1bbbfd6436d140d8aacd84ec"
server_key_version: "encryptionkey@0"
I20260619 08:27:37.146368 14015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:27:37.147228 14030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:37.147500 14015 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:37.147594 14015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "b466d5f9a39248cd94d770768c0c594f"
format_stamp: "Formatted at 2026-06-19 08:27:37 on dist-test-slave-5rhg"
server_key: "ad3e68a46806667b8dff05efb39ac548"
server_key_iv: "6fcbf34b1bbbfd6436d140d8aacd84ec"
server_key_version: "encryptionkey@0"
I20260619 08:27:37.147658 14015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:37.161427 14015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:37.163575 14015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:37.163704 14015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:37.167937 14015 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44995
I20260619 08:27:37.167999 14082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44995 every 8 connection(s)
I20260619 08:27:37.168442 14015 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:37.169389 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 14015
I20260619 08:27:37.169493 14083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:37.169533 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:37.169732 26110 external_mini_cluster.cc:1428] Setting key 8714428e422c4c51a7d52fc599b0ef62
I20260619 08:27:37.172521 14083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f: Bootstrap starting.
I20260619 08:27:37.173149 14083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:37.173437 14083 log.cc:824] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:37.174122 14083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f: No bootstrap required, opened a new log
I20260619 08:27:37.175618 14086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:37.170303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55533 (local address 127.0.0.1:44995)
0619 08:27:37.170513 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:37.170525 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:37.170565 (+ 40us) server_negotiation.cc:407] Connection header received
0619 08:27:37.171038 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:37.171045 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:37.171095 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:37.171255 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:37.171514 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:37.172516 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:37.173476 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:37.173976 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:37.174106 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:37.174956 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:37.174966 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:37.174966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:37.175142 (+ 176us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:37.175176 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:37.175188 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:37.175300 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:37.175390 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 08:27:37.175429 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":89,"threads_started":1}
I20260619 08:27:37.176227 14083 raft_consensus.cc:348] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b466d5f9a39248cd94d770768c0c594f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44995 } }
I20260619 08:27:37.176404 14083 raft_consensus.cc:374] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:37.176455 14083 raft_consensus.cc:729] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b466d5f9a39248cd94d770768c0c594f, State: Initialized, Role: FOLLOWER
I20260619 08:27:37.176584 14083 consensus_queue.cc:260] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b466d5f9a39248cd94d770768c0c594f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44995 } }
I20260619 08:27:37.176681 14083 raft_consensus.cc:388] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:37.176725 14083 raft_consensus.cc:482] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:37.176776 14083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:37.176789 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:27:37.177330 14083 raft_consensus.cc:504] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b466d5f9a39248cd94d770768c0c594f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44995 } }
I20260619 08:27:37.177445 14083 leader_election.cc:302] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b466d5f9a39248cd94d770768c0c594f; no voters:
I20260619 08:27:37.177649 14083 leader_election.cc:288] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:37.177681 14088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:37.177827 14088 raft_consensus.cc:686] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f [term 1 LEADER]: Becoming Leader. State: Replica: b466d5f9a39248cd94d770768c0c594f, State: Running, Role: LEADER
I20260619 08:27:37.177913 14088 consensus_queue.cc:237] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b466d5f9a39248cd94d770768c0c594f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44995 } }
I20260619 08:27:37.177937 14083 sys_catalog.cc:564] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:37.179373 14091 sys_catalog.cc:455] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b466d5f9a39248cd94d770768c0c594f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b466d5f9a39248cd94d770768c0c594f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44995 } } }
I20260619 08:27:37.179587 14093 sys_catalog.cc:455] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b466d5f9a39248cd94d770768c0c594f. Latest consensus state: current_term: 1 leader_uuid: "b466d5f9a39248cd94d770768c0c594f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b466d5f9a39248cd94d770768c0c594f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44995 } } }
I20260619 08:27:37.179630 14091 sys_catalog.cc:458] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:37.179641 14093 sys_catalog.cc:458] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:37.179916 14109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:37.181340 14109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:37.182366 14086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:37.178485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46265 (local address 127.0.0.1:44995)
0619 08:27:37.178520 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:37.178522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:37.178745 (+ 223us) server_negotiation.cc:407] Connection header received
0619 08:27:37.178814 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:37.178815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:37.178825 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:37.178930 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:37.179105 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:37.179981 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:37.180862 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:37.181196 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:37.181275 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:37.182092 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:37.182094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:37.182094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:37.182099 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:37.182117 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:37.182123 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:37.182189 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:37.182283 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 08:27:37.182295 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 08:27:37.183141 14109 catalog_manager.cc:1347] Generated new cluster ID: 62d64c34e1a446f69e486deea01dccb5
I20260619 08:27:37.183192 14109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:37.200749 14109 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:37.201377 14109 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:37.207104 14109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b466d5f9a39248cd94d770768c0c594f: Generated new TSK 0
I20260619 08:27:37.207329 14109 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:37.217412 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 14015
W20260619 08:27:37.221771 14000 connection.cc:535] client connection to 127.0.0.1:44995 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44995): Connection reset by peer (error 104)
2026-06-19T08:27:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1325 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:37 dist-test-slave-5rhg krb5kdc[14115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:37 dist-test-slave-5rhg krb5kdc[14115](info): set up 2 sockets
Jun 19 08:27:37 dist-test-slave-5rhg krb5kdc[14115](info): commencing operation
krb5kdc: starting...
W20260619 08:27:38.265240 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:38 dist-test-slave-5rhg krb5kdc[14115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:27:38.383802 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43171
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:38.466849 14128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:38.467116 14128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:38.467155 14128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:38.468405 14128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:38.468473 14128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:38.468497 14128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:38.468519 14128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:38.468546 14128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:38.470043 14128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43171
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:38.470335 14128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:38.470571 14128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:38.475442 14133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:38.475472 14136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:38.475659 14128 server_base.cc:1034] running on GCE node
W20260619 08:27:38.475500 14134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:38.476045 14128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:38.476413 14128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:38.477598 14128 hybrid_clock.cc:648] HybridClock initialized: now 1781857658477575 us; error 47 us; skew 500 ppm
Jun 19 08:27:38 dist-test-slave-5rhg krb5kdc[14115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857658, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:38.480463 14128 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:27:38.480973 14128 webserver.cc:460] Webserver started at http://127.0.0.1:35073/ using document root <none> and password file <none>
I20260619 08:27:38.481228 14128 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:38.481303 14128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:38.481465 14128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:38.482604 14128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "d8cd778c3a094188b6e6ce581ed50f44"
format_stamp: "Formatted at 2026-06-19 08:27:38 on dist-test-slave-5rhg"
server_key: "fbfae4ece7af6234a59a81734b45da12"
server_key_iv: "5d4987fac6787c703b3d29776f3cf705"
server_key_version: "encryptionkey@0"
I20260619 08:27:38.483006 14128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "d8cd778c3a094188b6e6ce581ed50f44"
format_stamp: "Formatted at 2026-06-19 08:27:38 on dist-test-slave-5rhg"
server_key: "fbfae4ece7af6234a59a81734b45da12"
server_key_iv: "5d4987fac6787c703b3d29776f3cf705"
server_key_version: "encryptionkey@0"
I20260619 08:27:38.484516 14128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:27:38.485335 14143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:38.485517 14128 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:27:38.485631 14128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "d8cd778c3a094188b6e6ce581ed50f44"
format_stamp: "Formatted at 2026-06-19 08:27:38 on dist-test-slave-5rhg"
server_key: "fbfae4ece7af6234a59a81734b45da12"
server_key_iv: "5d4987fac6787c703b3d29776f3cf705"
server_key_version: "encryptionkey@0"
I20260619 08:27:38.485714 14128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:38.489396 14128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:38.491549 14128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:38.491722 14128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:38.496755 14128 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43171
I20260619 08:27:38.496783 14195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43171 every 8 connection(s)
I20260619 08:27:38.497143 14128 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:38.498135 14196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:38.498574 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 14128
I20260619 08:27:38.498703 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:38.498888 26110 external_mini_cluster.cc:1428] Setting key d1d0cec6cd85481e8fb0ab59616ff038
I20260619 08:27:38.501665 14199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:38.499494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42047 (local address 127.0.0.1:43171)
0619 08:27:38.499684 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:38.499695 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:38.499729 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:27:38.500320 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:38.500328 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:38.500363 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:38.500526 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:38.501225 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:38.501229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:38.501229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:38.501234 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:38.501281 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:38.501296 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:38.501368 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:38.501436 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 08:27:38.501478 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":77,"threads_started":1}
I20260619 08:27:38.501823 14196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44: Bootstrap starting.
I20260619 08:27:38.502581 14196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:38.502722 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:27:38.502934 14196 log.cc:824] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:38.503629 14196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44: No bootstrap required, opened a new log
I20260619 08:27:38.505285 14199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:38.504304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50041 (local address 127.0.0.1:43171)
0619 08:27:38.504355 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:38.504357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:38.504392 (+ 35us) server_negotiation.cc:407] Connection header received
0619 08:27:38.504452 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:38.504453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:38.504457 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:38.504518 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:38.505064 (+ 546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:38.505066 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:38.505066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:38.505070 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:38.505085 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:38.505090 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:38.505154 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:38.505190 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 08:27:38.505202 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 08:27:38.506409 14196 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cd778c3a094188b6e6ce581ed50f44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43171 } }
I20260619 08:27:38.506599 14196 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:38.506649 14196 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8cd778c3a094188b6e6ce581ed50f44, State: Initialized, Role: FOLLOWER
I20260619 08:27:38.506775 14196 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cd778c3a094188b6e6ce581ed50f44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43171 } }
I20260619 08:27:38.506870 14196 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:38.506912 14196 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:38.506973 14196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:38.507527 14196 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cd778c3a094188b6e6ce581ed50f44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43171 } }
I20260619 08:27:38.507650 14196 leader_election.cc:302] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8cd778c3a094188b6e6ce581ed50f44; no voters:
I20260619 08:27:38.507815 14196 leader_election.cc:288] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:38.507851 14206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:38.508077 14206 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44 [term 1 LEADER]: Becoming Leader. State: Replica: d8cd778c3a094188b6e6ce581ed50f44, State: Running, Role: LEADER
I20260619 08:27:38.508104 14196 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:38.508231 14206 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cd778c3a094188b6e6ce581ed50f44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43171 } }
I20260619 08:27:38.508594 14207 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8cd778c3a094188b6e6ce581ed50f44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cd778c3a094188b6e6ce581ed50f44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43171 } } }
I20260619 08:27:38.508595 14208 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8cd778c3a094188b6e6ce581ed50f44. Latest consensus state: current_term: 1 leader_uuid: "d8cd778c3a094188b6e6ce581ed50f44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cd778c3a094188b6e6ce581ed50f44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43171 } } }
I20260619 08:27:38.508735 14207 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:38.508754 14208 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:38.508980 14213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:38.509500 14213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:38.513257 14213 catalog_manager.cc:1347] Generated new cluster ID: 3ac4626c86e449289334e8d10a2fcac2
I20260619 08:27:38.513302 14213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:38.530761 14213 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:38.530977 14213 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:38.546037 14213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8cd778c3a094188b6e6ce581ed50f44: Generated new TSK 0
I20260619 08:27:38.546288 14213 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:38.554670 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 14128
2026-06-19T08:27:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:38 dist-test-slave-5rhg krb5kdc[14227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:38 dist-test-slave-5rhg krb5kdc[14227](info): set up 2 sockets
Jun 19 08:27:38 dist-test-slave-5rhg krb5kdc[14227](info): commencing operation
krb5kdc: starting...
W20260619 08:27:39.589130 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:39 dist-test-slave-5rhg krb5kdc[14227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:27:39.706209 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42773
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:39.788066 14240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:39.788318 14240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:39.788343 14240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:39.789551 14240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:39.789598 14240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:39.789613 14240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:39.789626 14240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:39.789638 14240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:39.791221 14240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:39.791491 14240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:39.791730 14240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:39.796607 14245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:39.796622 14246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:39.796625 14240 server_base.cc:1034] running on GCE node
W20260619 08:27:39.796607 14248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:39.796993 14240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:39.797292 14240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:39.798456 14240 hybrid_clock.cc:648] HybridClock initialized: now 1781857659798437 us; error 41 us; skew 500 ppm
Jun 19 08:27:39 dist-test-slave-5rhg krb5kdc[14227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857659, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:39.801084 14240 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:27:39.801568 14240 webserver.cc:460] Webserver started at http://127.0.0.1:33111/ using document root <none> and password file <none>
I20260619 08:27:39.801795 14240 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:39.801860 14240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:39.802012 14240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:39.803072 14240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "e04814c9fc864fc4a70d3bc881bc2b78"
format_stamp: "Formatted at 2026-06-19 08:27:39 on dist-test-slave-5rhg"
server_key: "1b6a37335937ab7ed1c77415396cdf3f"
server_key_iv: "69311f517752a3c1811337b3e1247f0c"
server_key_version: "encryptionkey@0"
I20260619 08:27:39.803428 14240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "e04814c9fc864fc4a70d3bc881bc2b78"
format_stamp: "Formatted at 2026-06-19 08:27:39 on dist-test-slave-5rhg"
server_key: "1b6a37335937ab7ed1c77415396cdf3f"
server_key_iv: "69311f517752a3c1811337b3e1247f0c"
server_key_version: "encryptionkey@0"
I20260619 08:27:39.804852 14240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:27:39.805608 14255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:39.805776 14240 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:27:39.805879 14240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "e04814c9fc864fc4a70d3bc881bc2b78"
format_stamp: "Formatted at 2026-06-19 08:27:39 on dist-test-slave-5rhg"
server_key: "1b6a37335937ab7ed1c77415396cdf3f"
server_key_iv: "69311f517752a3c1811337b3e1247f0c"
server_key_version: "encryptionkey@0"
I20260619 08:27:39.805955 14240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:39.839654 14240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:39.841979 14240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:39.842159 14240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:39.846421 14240 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42773
I20260619 08:27:39.846464 14307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42773 every 8 connection(s)
I20260619 08:27:39.846863 14240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:39.847774 14308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:39.850324 14308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78: Bootstrap starting.
I20260619 08:27:39.850919 14308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:39.851182 14308 log.cc:824] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:39.851696 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 14240
I20260619 08:27:39.851830 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:39.852034 26110 external_mini_cluster.cc:1428] Setting key 31401d19731d8154fbed5e3f1346f515
I20260619 08:27:39.852041 14308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78: No bootstrap required, opened a new log
I20260619 08:27:39.854591 14308 raft_consensus.cc:348] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e04814c9fc864fc4a70d3bc881bc2b78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } }
I20260619 08:27:39.854799 14308 raft_consensus.cc:374] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:39.854843 14308 raft_consensus.cc:729] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e04814c9fc864fc4a70d3bc881bc2b78, State: Initialized, Role: FOLLOWER
I20260619 08:27:39.855027 14308 consensus_queue.cc:260] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e04814c9fc864fc4a70d3bc881bc2b78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } }
I20260619 08:27:39.855113 14308 raft_consensus.cc:388] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:39.855146 14308 raft_consensus.cc:482] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:39.855193 14308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:39.855989 14308 raft_consensus.cc:504] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e04814c9fc864fc4a70d3bc881bc2b78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } }
I20260619 08:27:39.856105 14308 leader_election.cc:302] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e04814c9fc864fc4a70d3bc881bc2b78; no voters:
I20260619 08:27:39.856303 14308 leader_election.cc:288] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:39.856359 14313 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:39.856508 14313 raft_consensus.cc:686] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78 [term 1 LEADER]: Becoming Leader. State: Replica: e04814c9fc864fc4a70d3bc881bc2b78, State: Running, Role: LEADER
I20260619 08:27:39.856572 14308 sys_catalog.cc:564] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:39.856631 14313 consensus_queue.cc:237] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e04814c9fc864fc4a70d3bc881bc2b78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } }
I20260619 08:27:39.857049 14314 sys_catalog.cc:455] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e04814c9fc864fc4a70d3bc881bc2b78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e04814c9fc864fc4a70d3bc881bc2b78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } } }
I20260619 08:27:39.857126 14314 sys_catalog.cc:458] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:39.857299 14315 sys_catalog.cc:455] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e04814c9fc864fc4a70d3bc881bc2b78. Latest consensus state: current_term: 1 leader_uuid: "e04814c9fc864fc4a70d3bc881bc2b78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e04814c9fc864fc4a70d3bc881bc2b78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } } }
I20260619 08:27:39.857354 14315 sys_catalog.cc:458] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:39.857749 14319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:39.858345 14319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:39.860080 14319 catalog_manager.cc:1347] Generated new cluster ID: b6783dd614ab4d48af6195256c150312
I20260619 08:27:39.860140 14319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:27:39 dist-test-slave-5rhg krb5kdc[14227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 08:27:39.866138 14312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:39.852607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43087 (local address 127.0.0.1:42773)
0619 08:27:39.852825 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:39.852838 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:39.852881 (+ 43us) server_negotiation.cc:407] Connection header received
0619 08:27:39.853702 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:39.853713 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:39.853785 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:39.853953 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:39.854570 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:39.856542 (+ 1972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:39.857902 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:39.858796 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:39.858919 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:39.862148 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:39.862152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:39.862152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:39.862166 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:39.864417 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:39.864836 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:39.864840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:39.864840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:39.864958 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:39.865177 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:39.865178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:39.865178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:39.865494 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:39.865650 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:39.865770 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 08:27:39.865917 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":92,"threads_started":1}
I20260619 08:27:39.867601 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:27:39.877489 14319 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:39.878836 14319 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:39.884301 14312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:39.869105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35177 (local address 127.0.0.1:42773)
0619 08:27:39.869142 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:39.869144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:39.877135 (+ 7991us) server_negotiation.cc:407] Connection header received
0619 08:27:39.877262 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:39.877264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:39.877275 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:39.877329 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:39.878159 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:39.878702 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:39.879575 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:39.879820 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:39.879902 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:39.881290 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:39.881291 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:39.881292 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:39.881300 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:39.882906 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:39.883317 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:39.883318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:39.883319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:39.883404 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:39.883646 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:39.883648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:39.883648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:39.883796 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:39.883913 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:39.884050 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 08:27:39.884176 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:27:39.887457 14319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e04814c9fc864fc4a70d3bc881bc2b78: Generated new TSK 0
I20260619 08:27:39.887642 14319 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:39.892664 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 14240
2026-06-19T08:27:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:39 dist-test-slave-5rhg krb5kdc[14339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:39 dist-test-slave-5rhg krb5kdc[14339](info): set up 2 sockets
Jun 19 08:27:39 dist-test-slave-5rhg krb5kdc[14339](info): commencing operation
krb5kdc: starting...
W20260619 08:27:40.925148 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:41 dist-test-slave-5rhg krb5kdc[14339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:27:41.041306 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44551
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:41.122083 14352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:41.122346 14352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:41.122396 14352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:41.123543 14352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:41.123613 14352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:41.123636 14352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:41.123656 14352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:41.123684 14352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:41.125118 14352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:41.125423 14352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:41.125656 14352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:41.130653 14360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:41.130648 14357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:41.130686 14358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:41.131026 14352 server_base.cc:1034] running on GCE node
I20260619 08:27:41.131320 14352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:41.131624 14352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:41.132776 14352 hybrid_clock.cc:648] HybridClock initialized: now 1781857661132763 us; error 32 us; skew 500 ppm
Jun 19 08:27:41 dist-test-slave-5rhg krb5kdc[14339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857661, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:41.135386 14352 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:27:41.135913 14352 webserver.cc:460] Webserver started at http://127.0.0.1:42887/ using document root <none> and password file <none>
I20260619 08:27:41.136185 14352 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:41.136270 14352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:41.136431 14352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:41.137552 14352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "a745720ccc7d45f7895a4e5a8170b236"
format_stamp: "Formatted at 2026-06-19 08:27:41 on dist-test-slave-5rhg"
server_key: "bd78131da9a1651dd50abfd6193fbf38"
server_key_iv: "b488cca21074f65ee44a47468a369e35"
server_key_version: "encryptionkey@0"
I20260619 08:27:41.137944 14352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "a745720ccc7d45f7895a4e5a8170b236"
format_stamp: "Formatted at 2026-06-19 08:27:41 on dist-test-slave-5rhg"
server_key: "bd78131da9a1651dd50abfd6193fbf38"
server_key_iv: "b488cca21074f65ee44a47468a369e35"
server_key_version: "encryptionkey@0"
I20260619 08:27:41.139350 14352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:27:41.140123 14367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:41.140331 14352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:41.140437 14352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "a745720ccc7d45f7895a4e5a8170b236"
format_stamp: "Formatted at 2026-06-19 08:27:41 on dist-test-slave-5rhg"
server_key: "bd78131da9a1651dd50abfd6193fbf38"
server_key_iv: "b488cca21074f65ee44a47468a369e35"
server_key_version: "encryptionkey@0"
I20260619 08:27:41.140525 14352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:41.159219 14352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:41.161849 14352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:41.162041 14352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:41.166270 14352 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44551
I20260619 08:27:41.166311 14419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44551 every 8 connection(s)
I20260619 08:27:41.166697 14352 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:41.167670 14420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:41.170051 14420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236: Bootstrap starting.
I20260619 08:27:41.170657 14420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:41.170953 14420 log.cc:824] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:41.171684 14420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236: No bootstrap required, opened a new log
I20260619 08:27:41.173365 14420 raft_consensus.cc:348] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a745720ccc7d45f7895a4e5a8170b236" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44551 } }
I20260619 08:27:41.173514 14420 raft_consensus.cc:374] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:41.173538 14420 raft_consensus.cc:729] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a745720ccc7d45f7895a4e5a8170b236, State: Initialized, Role: FOLLOWER
I20260619 08:27:41.173700 14420 consensus_queue.cc:260] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a745720ccc7d45f7895a4e5a8170b236" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44551 } }
I20260619 08:27:41.173765 14420 raft_consensus.cc:388] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:41.173786 14420 raft_consensus.cc:482] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:41.173837 14420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:41.174369 14420 raft_consensus.cc:504] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a745720ccc7d45f7895a4e5a8170b236" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44551 } }
I20260619 08:27:41.174484 14420 leader_election.cc:302] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a745720ccc7d45f7895a4e5a8170b236; no voters:
I20260619 08:27:41.174664 14420 leader_election.cc:288] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:41.174844 14423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:41.175033 14423 raft_consensus.cc:686] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236 [term 1 LEADER]: Becoming Leader. State: Replica: a745720ccc7d45f7895a4e5a8170b236, State: Running, Role: LEADER
I20260619 08:27:41.175109 14420 sys_catalog.cc:564] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:41.175192 14423 consensus_queue.cc:237] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a745720ccc7d45f7895a4e5a8170b236" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44551 } }
I20260619 08:27:41.175596 14424 sys_catalog.cc:455] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a745720ccc7d45f7895a4e5a8170b236" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a745720ccc7d45f7895a4e5a8170b236" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44551 } } }
I20260619 08:27:41.175707 14424 sys_catalog.cc:458] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:41.175599 14425 sys_catalog.cc:455] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a745720ccc7d45f7895a4e5a8170b236. Latest consensus state: current_term: 1 leader_uuid: "a745720ccc7d45f7895a4e5a8170b236" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a745720ccc7d45f7895a4e5a8170b236" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44551 } } }
I20260619 08:27:41.175838 14425 sys_catalog.cc:458] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:41.175964 14427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:41.176409 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 14352
I20260619 08:27:41.176530 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:41.176574 14427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:41.176723 26110 external_mini_cluster.cc:1428] Setting key 97523937838b4f37ff2095fc33159512
I20260619 08:27:41.182075 14427 catalog_manager.cc:1347] Generated new cluster ID: 7d0af5af36ef4ac39ad51256b9191dd3
I20260619 08:27:41.182179 14427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:41.182438 14442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:41.177507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49177 (local address 127.0.0.1:44551)
0619 08:27:41.177815 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:41.177825 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:41.177856 (+ 31us) server_negotiation.cc:407] Connection header received
0619 08:27:41.177978 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:41.177985 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:41.178044 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:41.178207 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:41.178385 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:41.179291 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:41.180096 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:41.180696 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:41.180825 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:41.181602 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:41.181607 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:41.181607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:41.181818 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:41.181860 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:41.181875 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:41.182137 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:41.182181 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 08:27:41.182253 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":214,"threads_started":1}
I20260619 08:27:41.183323 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:27:41.187556 14442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:41.184651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53817 (local address 127.0.0.1:44551)
0619 08:27:41.184686 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:41.184688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:41.184880 (+ 192us) server_negotiation.cc:407] Connection header received
0619 08:27:41.184951 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:41.184952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:41.184963 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:41.185015 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:41.185133 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:41.185608 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:41.186320 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:41.186540 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:41.186615 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:41.187312 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:41.187313 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:41.187314 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:41.187318 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:41.187333 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:41.187337 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:41.187384 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:41.187447 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 08:27:41.187470 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 08:27:41.207649 14427 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:41.208280 14427 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:41.226588 14427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a745720ccc7d45f7895a4e5a8170b236: Generated new TSK 0
I20260619 08:27:41.226817 14427 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:41.237183 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 14352
2026-06-19T08:27:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1344 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:41 dist-test-slave-5rhg krb5kdc[14451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:41 dist-test-slave-5rhg krb5kdc[14451](info): set up 2 sockets
Jun 19 08:27:41 dist-test-slave-5rhg krb5kdc[14451](info): commencing operation
krb5kdc: starting...
W20260619 08:27:42.305222 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:42 dist-test-slave-5rhg krb5kdc[14451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:27:42.421085 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39691
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:42.503561 14464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:42.503827 14464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:42.503866 14464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:42.505107 14464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:42.505185 14464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:42.505210 14464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:42.505230 14464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:42.505261 14464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:42.506807 14464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:42.507099 14464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:42.507330 14464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:42.513183 14470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:42.513191 14469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:42.513195 14472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:42.513635 14464 server_base.cc:1034] running on GCE node
I20260619 08:27:42.513893 14464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:42.514329 14464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:42.515522 14464 hybrid_clock.cc:648] HybridClock initialized: now 1781857662515492 us; error 39 us; skew 500 ppm
Jun 19 08:27:42 dist-test-slave-5rhg krb5kdc[14451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857662, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:42.525810 14464 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:27:42.526391 14464 webserver.cc:460] Webserver started at http://127.0.0.1:44207/ using document root <none> and password file <none>
I20260619 08:27:42.526640 14464 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:42.526710 14464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:42.526870 14464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:42.527994 14464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "309930a403714a20a2ae12d05ec13a53"
format_stamp: "Formatted at 2026-06-19 08:27:42 on dist-test-slave-5rhg"
server_key: "a54d19888876f3872c12df162ae2f376"
server_key_iv: "bf23a40699b83e0ee894f71d62d86bb4"
server_key_version: "encryptionkey@0"
I20260619 08:27:42.528406 14464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "309930a403714a20a2ae12d05ec13a53"
format_stamp: "Formatted at 2026-06-19 08:27:42 on dist-test-slave-5rhg"
server_key: "a54d19888876f3872c12df162ae2f376"
server_key_iv: "bf23a40699b83e0ee894f71d62d86bb4"
server_key_version: "encryptionkey@0"
I20260619 08:27:42.529796 14464 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:27:42.530561 14479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:42.530757 14464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:27:42.530867 14464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "309930a403714a20a2ae12d05ec13a53"
format_stamp: "Formatted at 2026-06-19 08:27:42 on dist-test-slave-5rhg"
server_key: "a54d19888876f3872c12df162ae2f376"
server_key_iv: "bf23a40699b83e0ee894f71d62d86bb4"
server_key_version: "encryptionkey@0"
I20260619 08:27:42.530948 14464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:42.534461 14464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:42.536870 14464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:42.537039 14464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:42.541061 14464 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39691
I20260619 08:27:42.541102 14531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39691 every 8 connection(s)
I20260619 08:27:42.541455 14464 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:42.542466 14532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:42.544932 14532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53: Bootstrap starting.
I20260619 08:27:42.545547 14532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:42.545876 14532 log.cc:824] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:42.546063 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 14464
I20260619 08:27:42.546196 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:42.546427 26110 external_mini_cluster.cc:1428] Setting key 8f6733a2a25cd9ad0638f53c00c8d95c
I20260619 08:27:42.546677 14532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53: No bootstrap required, opened a new log
I20260619 08:27:42.548978 14532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309930a403714a20a2ae12d05ec13a53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39691 } }
I20260619 08:27:42.549145 14532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:42.549216 14532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 309930a403714a20a2ae12d05ec13a53, State: Initialized, Role: FOLLOWER
I20260619 08:27:42.549366 14532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309930a403714a20a2ae12d05ec13a53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39691 } }
I20260619 08:27:42.549470 14532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:42.549512 14532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:42.549566 14532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:42.549649 14536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:42.546969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41717 (local address 127.0.0.1:39691)
0619 08:27:42.547151 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:42.547160 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:42.547195 (+ 35us) server_negotiation.cc:407] Connection header received
0619 08:27:42.548014 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:42.548021 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:42.548050 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:42.548244 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:42.548985 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:42.548989 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:42.548990 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:42.548994 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:42.549029 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:42.549040 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:42.549118 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:42.549188 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 08:27:42.549233 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260619 08:27:42.550395 14532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309930a403714a20a2ae12d05ec13a53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39691 } }
I20260619 08:27:42.550525 14532 leader_election.cc:302] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 309930a403714a20a2ae12d05ec13a53; no voters:
I20260619 08:27:42.550642 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:27:42.550694 14532 leader_election.cc:288] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:42.550730 14537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:42.550884 14537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53 [term 1 LEADER]: Becoming Leader. State: Replica: 309930a403714a20a2ae12d05ec13a53, State: Running, Role: LEADER
I20260619 08:27:42.551002 14532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:42.551019 14537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309930a403714a20a2ae12d05ec13a53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39691 } }
I20260619 08:27:42.551415 14538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "309930a403714a20a2ae12d05ec13a53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309930a403714a20a2ae12d05ec13a53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39691 } } }
I20260619 08:27:42.551576 14538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:42.551396 14537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 309930a403714a20a2ae12d05ec13a53. Latest consensus state: current_term: 1 leader_uuid: "309930a403714a20a2ae12d05ec13a53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309930a403714a20a2ae12d05ec13a53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39691 } } }
I20260619 08:27:42.551774 14537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:42.551981 14548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:42.552582 14548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:42.553694 14536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:42.552410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43855 (local address 127.0.0.1:39691)
0619 08:27:42.552449 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:42.552451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:42.552599 (+ 148us) server_negotiation.cc:407] Connection header received
0619 08:27:42.552697 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:42.552698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:42.552703 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:42.552732 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:42.553459 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:42.553461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:42.553461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:42.553465 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:42.553480 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:42.553484 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:42.553513 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:42.553619 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 08:27:42.553632 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:27:42.554776 14548 catalog_manager.cc:1347] Generated new cluster ID: 4721a0534d8d4a3f9fe5669efd709a2d
I20260619 08:27:42.554829 14548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:42.567361 14548 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:42.567561 14548 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:42.573379 14548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 309930a403714a20a2ae12d05ec13a53: Generated new TSK 0
I20260619 08:27:42.573602 14548 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:42.577252 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 14464
2026-06-19T08:27:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1339 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:42 dist-test-slave-5rhg krb5kdc[14563](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:42 dist-test-slave-5rhg krb5kdc[14563](info): set up 2 sockets
Jun 19 08:27:42 dist-test-slave-5rhg krb5kdc[14563](info): commencing operation
krb5kdc: starting...
W20260619 08:27:43.609144 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:43 dist-test-slave-5rhg krb5kdc[14563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:27:43.724826 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39939
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:43.806159 14576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:43.806412 14576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:43.806473 14576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:43.807662 14576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:43.807734 14576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:43.807760 14576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:43.807788 14576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:43.807809 14576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:43.809343 14576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:43.809620 14576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:43.809854 14576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:43.814728 14582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:43.814778 14581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:43.814790 14584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:43.815057 14576 server_base.cc:1034] running on GCE node
I20260619 08:27:43.815346 14576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:43.815644 14576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:43.816802 14576 hybrid_clock.cc:648] HybridClock initialized: now 1781857663816770 us; error 40 us; skew 500 ppm
Jun 19 08:27:43 dist-test-slave-5rhg krb5kdc[14563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857663, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:43.819414 14576 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:27:43.819959 14576 webserver.cc:460] Webserver started at http://127.0.0.1:43101/ using document root <none> and password file <none>
I20260619 08:27:43.820221 14576 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:43.820271 14576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:43.820364 14576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:43.821388 14576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "0cd62a35522a4595b1d7e4a70866d3c5"
format_stamp: "Formatted at 2026-06-19 08:27:43 on dist-test-slave-5rhg"
server_key: "43bc7342d9785bced1f64a1a4f9df7d8"
server_key_iv: "6a73dca78422f915d19a5e599a32c2bd"
server_key_version: "encryptionkey@0"
I20260619 08:27:43.821700 14576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "0cd62a35522a4595b1d7e4a70866d3c5"
format_stamp: "Formatted at 2026-06-19 08:27:43 on dist-test-slave-5rhg"
server_key: "43bc7342d9785bced1f64a1a4f9df7d8"
server_key_iv: "6a73dca78422f915d19a5e599a32c2bd"
server_key_version: "encryptionkey@0"
I20260619 08:27:43.823017 14576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:27:43.823899 14591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:43.824097 14576 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:27:43.824213 14576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "0cd62a35522a4595b1d7e4a70866d3c5"
format_stamp: "Formatted at 2026-06-19 08:27:43 on dist-test-slave-5rhg"
server_key: "43bc7342d9785bced1f64a1a4f9df7d8"
server_key_iv: "6a73dca78422f915d19a5e599a32c2bd"
server_key_version: "encryptionkey@0"
I20260619 08:27:43.824301 14576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:43.860801 14576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:43.863270 14576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:43.863441 14576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:43.867589 14576 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39939
I20260619 08:27:43.867625 14643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39939 every 8 connection(s)
I20260619 08:27:43.868028 14576 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:43.868970 14644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:43.869905 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 14576
I20260619 08:27:43.870031 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:43.870245 26110 external_mini_cluster.cc:1428] Setting key 69965968f35271e4fbdc603065b7ddf2
I20260619 08:27:43.872568 14644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5: Bootstrap starting.
I20260619 08:27:43.873194 14644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:43.873512 14644 log.cc:824] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:43.874238 14644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5: No bootstrap required, opened a new log
I20260619 08:27:43.876762 14644 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd62a35522a4595b1d7e4a70866d3c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39939 } }
I20260619 08:27:43.876957 14644 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:43.876986 14644 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd62a35522a4595b1d7e4a70866d3c5, State: Initialized, Role: FOLLOWER
I20260619 08:27:43.877136 14644 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd62a35522a4595b1d7e4a70866d3c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39939 } }
I20260619 08:27:43.877204 14644 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:43.877226 14644 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:43.877292 14644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5 [term 0 FOLLOWER]: Advancing to term 1
Jun 19 08:27:43 dist-test-slave-5rhg krb5kdc[14563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 08:27:43.877899 14644 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd62a35522a4595b1d7e4a70866d3c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39939 } }
I20260619 08:27:43.878017 14644 leader_election.cc:302] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cd62a35522a4595b1d7e4a70866d3c5; no voters:
I20260619 08:27:43.878204 14644 leader_election.cc:288] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:43.878249 14649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:43.878373 14649 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5 [term 1 LEADER]: Becoming Leader. State: Replica: 0cd62a35522a4595b1d7e4a70866d3c5, State: Running, Role: LEADER
I20260619 08:27:43.878486 14644 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:43.878508 14649 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd62a35522a4595b1d7e4a70866d3c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39939 } }
I20260619 08:27:43.878947 14649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cd62a35522a4595b1d7e4a70866d3c5. Latest consensus state: current_term: 1 leader_uuid: "0cd62a35522a4595b1d7e4a70866d3c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd62a35522a4595b1d7e4a70866d3c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39939 } } }
I20260619 08:27:43.878963 14651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cd62a35522a4595b1d7e4a70866d3c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cd62a35522a4595b1d7e4a70866d3c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39939 } } }
I20260619 08:27:43.879130 14649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:43.879143 14651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:43.879359 14658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:43.879925 14658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:43.881745 14658 catalog_manager.cc:1347] Generated new cluster ID: 35609886a0a0419ab0ac72629af651d7
I20260619 08:27:43.881795 14658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:43.882066 14647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:43.870841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:51387 (local address 127.0.0.1:39939)
0619 08:27:43.871023 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:43.871032 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:43.871076 (+ 44us) server_negotiation.cc:407] Connection header received
0619 08:27:43.871541 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:43.871548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:43.871600 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:43.871723 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:43.872547 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:43.873599 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:43.874602 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:43.875040 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:43.875120 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:43.878054 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:43.878058 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:43.878058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:43.878078 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:43.880277 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:43.880726 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:43.880729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:43.880729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:43.880817 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:43.881083 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:43.881085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:43.881085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:43.881409 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:43.881569 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:43.881758 (+ 189us) server_negotiation.cc:299] Negotiation successful
0619 08:27:43.881870 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260619 08:27:43.883067 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:27:43.891790 14647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:43.884567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:49995 (local address 127.0.0.1:39939)
0619 08:27:43.884608 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:43.884610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:43.884718 (+ 108us) server_negotiation.cc:407] Connection header received
0619 08:27:43.884747 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:43.884748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:43.884758 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:43.884825 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:43.885389 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:43.886141 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:43.887118 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:43.887455 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:43.887525 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:43.888963 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:43.888964 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:43.888965 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:43.888972 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:43.890553 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:43.890987 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:43.890988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:43.890988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:43.891050 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:43.891279 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:43.891280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:43.891281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:43.891412 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:43.891528 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:43.891634 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 08:27:43.891696 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":20}
I20260619 08:27:43.894110 14658 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:43.894708 14658 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:43.902140 14658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cd62a35522a4595b1d7e4a70866d3c5: Generated new TSK 0
I20260619 08:27:43.902374 14658 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:43.906489 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 14576
W20260619 08:27:43.910440 14561 connection.cc:535] client connection to 127.0.0.1:39939 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39939): Connection reset by peer (error 104)
2026-06-19T08:27:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:43 dist-test-slave-5rhg krb5kdc[14675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:43 dist-test-slave-5rhg krb5kdc[14675](info): set up 2 sockets
Jun 19 08:27:43 dist-test-slave-5rhg krb5kdc[14675](info): commencing operation
krb5kdc: starting...
W20260619 08:27:44.949134 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:45 dist-test-slave-5rhg krb5kdc[14675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:27:45.064674 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34467
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:45.146098 14688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:45.146376 14688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:45.146432 14688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:45.147643 14688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:45.147722 14688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:45.147746 14688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:45.147766 14688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:45.147784 14688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:45.149334 14688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:45.149631 14688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:45.149878 14688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:45.154896 14696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:45.154887 14693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:45.154976 14694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:45.155155 14688 server_base.cc:1034] running on GCE node
I20260619 08:27:45.155453 14688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:45.155761 14688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:45.156953 14688 hybrid_clock.cc:648] HybridClock initialized: now 1781857665156944 us; error 36 us; skew 500 ppm
Jun 19 08:27:45 dist-test-slave-5rhg krb5kdc[14675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857665, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:45.159575 14688 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:27:45.160077 14688 webserver.cc:460] Webserver started at http://127.0.0.1:37103/ using document root <none> and password file <none>
I20260619 08:27:45.160357 14688 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:45.160430 14688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:45.160581 14688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:45.161688 14688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "8e369b55036d405b8c40485a350e471f"
format_stamp: "Formatted at 2026-06-19 08:27:45 on dist-test-slave-5rhg"
server_key: "b3fd0891706e42ef72c4489b96ba4a18"
server_key_iv: "36c03010f2b312d0287d24e1fcaa48a2"
server_key_version: "encryptionkey@0"
I20260619 08:27:45.162060 14688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "8e369b55036d405b8c40485a350e471f"
format_stamp: "Formatted at 2026-06-19 08:27:45 on dist-test-slave-5rhg"
server_key: "b3fd0891706e42ef72c4489b96ba4a18"
server_key_iv: "36c03010f2b312d0287d24e1fcaa48a2"
server_key_version: "encryptionkey@0"
I20260619 08:27:45.163398 14688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:27:45.164119 14703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:45.164419 14688 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:45.164526 14688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "8e369b55036d405b8c40485a350e471f"
format_stamp: "Formatted at 2026-06-19 08:27:45 on dist-test-slave-5rhg"
server_key: "b3fd0891706e42ef72c4489b96ba4a18"
server_key_iv: "36c03010f2b312d0287d24e1fcaa48a2"
server_key_version: "encryptionkey@0"
I20260619 08:27:45.164603 14688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:45.182168 14688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:45.184773 14688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:45.184955 14688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:45.189244 14688 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34467
I20260619 08:27:45.189322 14755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34467 every 8 connection(s)
I20260619 08:27:45.189698 14688 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:45.190559 14756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:45.193080 14756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f: Bootstrap starting.
I20260619 08:27:45.193688 14756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:45.193991 14756 log.cc:824] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:45.194751 14756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f: No bootstrap required, opened a new log
I20260619 08:27:45.196395 14756 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e369b55036d405b8c40485a350e471f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34467 } }
I20260619 08:27:45.196542 14756 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:45.196565 14756 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e369b55036d405b8c40485a350e471f, State: Initialized, Role: FOLLOWER
I20260619 08:27:45.196743 14756 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e369b55036d405b8c40485a350e471f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34467 } }
I20260619 08:27:45.196806 14756 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:45.196830 14756 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:45.196903 14756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:45.197455 14756 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e369b55036d405b8c40485a350e471f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34467 } }
I20260619 08:27:45.197571 14756 leader_election.cc:302] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e369b55036d405b8c40485a350e471f; no voters:
I20260619 08:27:45.197759 14756 leader_election.cc:288] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:45.197817 14759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:45.197991 14759 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f [term 1 LEADER]: Becoming Leader. State: Replica: 8e369b55036d405b8c40485a350e471f, State: Running, Role: LEADER
I20260619 08:27:45.198065 14756 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:45.198163 14759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e369b55036d405b8c40485a350e471f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34467 } }
I20260619 08:27:45.198801 14760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e369b55036d405b8c40485a350e471f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e369b55036d405b8c40485a350e471f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34467 } } }
I20260619 08:27:45.198812 14761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e369b55036d405b8c40485a350e471f. Latest consensus state: current_term: 1 leader_uuid: "8e369b55036d405b8c40485a350e471f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e369b55036d405b8c40485a350e471f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34467 } } }
I20260619 08:27:45.198916 14760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:45.198928 14761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:45.199113 14768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:45.199599 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 14688
I20260619 08:27:45.199625 14768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:45.199725 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:45.199928 26110 external_mini_cluster.cc:1428] Setting key 99d722bb5a4468c558ee62b1bc906032
I20260619 08:27:45.201793 14768 catalog_manager.cc:1347] Generated new cluster ID: 9a907fc7a0e84f7e8685aac10ed5718b
I20260619 08:27:45.201846 14768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:45.205575 14778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:45.200532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:53727 (local address 127.0.0.1:34467)
0619 08:27:45.200716 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:45.200726 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:45.201016 (+ 290us) server_negotiation.cc:407] Connection header received
0619 08:27:45.201177 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:45.201184 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:45.201233 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:45.201356 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:45.201638 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:45.202681 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:45.203519 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:45.203973 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:45.204095 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:45.204895 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:45.204899 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:45.204900 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:45.205081 (+ 181us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:45.205112 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:45.205124 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:45.205220 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:45.205338 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 08:27:45.205372 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":96,"threads_started":1}
I20260619 08:27:45.206543 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:27:45.210822 14778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:45.208034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:59033 (local address 127.0.0.1:34467)
0619 08:27:45.208071 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:45.208073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:45.208081 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:27:45.208135 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:45.208136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:45.208167 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:45.208210 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:45.208345 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:45.208819 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:45.209540 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:45.209766 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:45.209855 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:45.210578 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:45.210580 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:45.210580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:45.210585 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:45.210599 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:45.210603 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:45.210656 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:45.210713 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 08:27:45.210725 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:27:45.211987 14768 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:45.212558 14768 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:45.221757 14768 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e369b55036d405b8c40485a350e471f: Generated new TSK 0
I20260619 08:27:45.221953 14768 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:45.226395 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 14688
W20260619 08:27:45.230705 14673 connection.cc:535] client connection to 127.0.0.1:34467 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34467): Connection reset by peer (error 104)
2026-06-19T08:27:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:45 dist-test-slave-5rhg krb5kdc[14787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:45 dist-test-slave-5rhg krb5kdc[14787](info): set up 2 sockets
Jun 19 08:27:45 dist-test-slave-5rhg krb5kdc[14787](info): commencing operation
krb5kdc: starting...
W20260619 08:27:46.329216 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:46 dist-test-slave-5rhg krb5kdc[14787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:27:46.442575 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35399
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:46.523126 14800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:46.523380 14800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:46.523422 14800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:46.524607 14800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:46.524682 14800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:46.524708 14800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:46.524734 14800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:46.524789 14800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:46.526995 14800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:46.527271 14800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:46.527500 14800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:46.532500 14805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:46.532513 14806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:46.532513 14808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:46.532697 14800 server_base.cc:1034] running on GCE node
I20260619 08:27:46.532987 14800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:46.533319 14800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:46.534466 14800 hybrid_clock.cc:648] HybridClock initialized: now 1781857666534455 us; error 32 us; skew 500 ppm
Jun 19 08:27:46 dist-test-slave-5rhg krb5kdc[14787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857666, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:46.537125 14800 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:27:46.537595 14800 webserver.cc:460] Webserver started at http://127.0.0.1:38881/ using document root <none> and password file <none>
I20260619 08:27:46.537784 14800 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:46.537827 14800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:46.537914 14800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:46.538910 14800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "ce4cc84a445c4455a397d408ad143686"
format_stamp: "Formatted at 2026-06-19 08:27:46 on dist-test-slave-5rhg"
server_key: "3e90e92c71ca9602f60d62039f185a50"
server_key_iv: "641c30febb924b04bf8c2dd21445abe6"
server_key_version: "encryptionkey@0"
I20260619 08:27:46.539204 14800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "ce4cc84a445c4455a397d408ad143686"
format_stamp: "Formatted at 2026-06-19 08:27:46 on dist-test-slave-5rhg"
server_key: "3e90e92c71ca9602f60d62039f185a50"
server_key_iv: "641c30febb924b04bf8c2dd21445abe6"
server_key_version: "encryptionkey@0"
I20260619 08:27:46.540548 14800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:27:46.541294 14815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:46.541496 14800 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:27:46.541568 14800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "ce4cc84a445c4455a397d408ad143686"
format_stamp: "Formatted at 2026-06-19 08:27:46 on dist-test-slave-5rhg"
server_key: "3e90e92c71ca9602f60d62039f185a50"
server_key_iv: "641c30febb924b04bf8c2dd21445abe6"
server_key_version: "encryptionkey@0"
I20260619 08:27:46.541664 14800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:46.544984 14800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:46.547274 14800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:46.547433 14800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:46.551472 14800 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35399
I20260619 08:27:46.551501 14867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35399 every 8 connection(s)
I20260619 08:27:46.551853 14800 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:46.552850 14868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:46.555059 14868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686: Bootstrap starting.
I20260619 08:27:46.555629 14868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:46.555913 14868 log.cc:824] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:46.556667 14868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686: No bootstrap required, opened a new log
I20260619 08:27:46.557013 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 14800
I20260619 08:27:46.557138 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:46.557334 26110 external_mini_cluster.cc:1428] Setting key 14bac3065be0bc28dc274829b532707a
I20260619 08:27:46.558959 14868 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4cc84a445c4455a397d408ad143686" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35399 } }
I20260619 08:27:46.559129 14868 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:46.559173 14868 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce4cc84a445c4455a397d408ad143686, State: Initialized, Role: FOLLOWER
I20260619 08:27:46.559330 14868 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4cc84a445c4455a397d408ad143686" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35399 } }
I20260619 08:27:46.559437 14868 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:46.559480 14868 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:46.559548 14868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:46.559798 14872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:46.557821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:57677 (local address 127.0.0.1:35399)
0619 08:27:46.558005 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:46.558012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:46.558038 (+ 26us) server_negotiation.cc:407] Connection header received
0619 08:27:46.558473 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:46.558480 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:46.558503 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:46.558622 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:46.559248 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:46.559252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:46.559252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:46.559256 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:46.559309 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:46.559322 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:46.559386 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:46.559551 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 08:27:46.559596 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260619 08:27:46.560549 14868 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4cc84a445c4455a397d408ad143686" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35399 } }
I20260619 08:27:46.560680 14868 leader_election.cc:302] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce4cc84a445c4455a397d408ad143686; no voters:
I20260619 08:27:46.560842 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:27:46.560855 14868 leader_election.cc:288] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:46.560884 14873 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:46.561070 14873 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686 [term 1 LEADER]: Becoming Leader. State: Replica: ce4cc84a445c4455a397d408ad143686, State: Running, Role: LEADER
I20260619 08:27:46.561084 14868 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:46.561254 14873 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4cc84a445c4455a397d408ad143686" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35399 } }
I20260619 08:27:46.561575 14875 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce4cc84a445c4455a397d408ad143686" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4cc84a445c4455a397d408ad143686" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35399 } } }
I20260619 08:27:46.561642 14875 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:46.561909 14879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:46.562403 14879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:46.562891 14874 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce4cc84a445c4455a397d408ad143686. Latest consensus state: current_term: 1 leader_uuid: "ce4cc84a445c4455a397d408ad143686" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4cc84a445c4455a397d408ad143686" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35399 } } }
I20260619 08:27:46.562975 14874 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:46.563743 14872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:46.562452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:36707 (local address 127.0.0.1:35399)
0619 08:27:46.562489 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:46.562491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:46.562725 (+ 234us) server_negotiation.cc:407] Connection header received
0619 08:27:46.562846 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:46.562847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:46.562854 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:46.562912 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:46.563521 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:46.563522 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:46.563522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:46.563526 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:46.563541 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:46.563545 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:46.563604 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:46.563676 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 08:27:46.563688 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:27:46.564250 14879 catalog_manager.cc:1347] Generated new cluster ID: 943f398af36a46268544ee63373e9a25
I20260619 08:27:46.564319 14879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:46.582698 14879 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:46.582942 14879 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:46.586995 14879 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce4cc84a445c4455a397d408ad143686: Generated new TSK 0
I20260619 08:27:46.587249 14879 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:46.596859 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 14800
2026-06-19T08:27:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1370 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:46 dist-test-slave-5rhg krb5kdc[14899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:46 dist-test-slave-5rhg krb5kdc[14899](info): set up 2 sockets
Jun 19 08:27:46 dist-test-slave-5rhg krb5kdc[14899](info): commencing operation
krb5kdc: starting...
W20260619 08:27:47.641134 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:47 dist-test-slave-5rhg krb5kdc[14899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:27:47.758756 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43469
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:47.841607 14912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:47.841876 14912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:47.841921 14912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:47.843111 14912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:47.843180 14912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:47.843215 14912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:47.843247 14912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:47.843272 14912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:47.844833 14912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43469
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:47.845129 14912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:47.845378 14912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:27:47.850427 14912 server_base.cc:1034] running on GCE node
W20260619 08:27:47.850329 14920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:47.850443 14918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:47.850329 14917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:47.850847 14912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:47.851208 14912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:47.852380 14912 hybrid_clock.cc:648] HybridClock initialized: now 1781857667852363 us; error 28 us; skew 500 ppm
Jun 19 08:27:47 dist-test-slave-5rhg krb5kdc[14899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857667, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:47.859234 14912 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:27:47.859743 14912 webserver.cc:460] Webserver started at http://127.0.0.1:39089/ using document root <none> and password file <none>
I20260619 08:27:47.859979 14912 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:47.860049 14912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:47.860247 14912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:47.861394 14912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "467059f8f55e47c7a24a6a81d104575c"
format_stamp: "Formatted at 2026-06-19 08:27:47 on dist-test-slave-5rhg"
server_key: "af4c662248e03fb965ed7a2ddab8e7f7"
server_key_iv: "def7d5434f3f847b46c59475b7bb3f9b"
server_key_version: "encryptionkey@0"
I20260619 08:27:47.861768 14912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "467059f8f55e47c7a24a6a81d104575c"
format_stamp: "Formatted at 2026-06-19 08:27:47 on dist-test-slave-5rhg"
server_key: "af4c662248e03fb965ed7a2ddab8e7f7"
server_key_iv: "def7d5434f3f847b46c59475b7bb3f9b"
server_key_version: "encryptionkey@0"
I20260619 08:27:47.863204 14912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:27:47.864022 14927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:47.864259 14912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:47.864360 14912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "467059f8f55e47c7a24a6a81d104575c"
format_stamp: "Formatted at 2026-06-19 08:27:47 on dist-test-slave-5rhg"
server_key: "af4c662248e03fb965ed7a2ddab8e7f7"
server_key_iv: "def7d5434f3f847b46c59475b7bb3f9b"
server_key_version: "encryptionkey@0"
I20260619 08:27:47.864444 14912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:47.875161 14912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:47.877693 14912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:47.877887 14912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:47.881996 14912 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43469
I20260619 08:27:47.882064 14979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43469 every 8 connection(s)
I20260619 08:27:47.882417 14912 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:47.883302 14980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:47.884032 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 14912
I20260619 08:27:47.884188 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:47.884394 26110 external_mini_cluster.cc:1428] Setting key 85664c0862ca15934fc75007f092cddd
I20260619 08:27:47.886631 14980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c: Bootstrap starting.
I20260619 08:27:47.887244 14980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:47.887540 14980 log.cc:824] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:47.888271 14980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c: No bootstrap required, opened a new log
I20260619 08:27:47.890172 14980 raft_consensus.cc:348] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467059f8f55e47c7a24a6a81d104575c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43469 } }
I20260619 08:27:47.890350 14980 raft_consensus.cc:374] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:47.890401 14980 raft_consensus.cc:729] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 467059f8f55e47c7a24a6a81d104575c, State: Initialized, Role: FOLLOWER
I20260619 08:27:47.890537 14980 consensus_queue.cc:260] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467059f8f55e47c7a24a6a81d104575c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43469 } }
I20260619 08:27:47.890630 14980 raft_consensus.cc:388] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:47.890690 14980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:47.890744 14980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c [term 0 FOLLOWER]: Advancing to term 1
Jun 19 08:27:47 dist-test-slave-5rhg krb5kdc[14899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 08:27:47.891384 14980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467059f8f55e47c7a24a6a81d104575c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43469 } }
I20260619 08:27:47.891503 14980 leader_election.cc:302] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 467059f8f55e47c7a24a6a81d104575c; no voters:
I20260619 08:27:47.891695 14980 leader_election.cc:288] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:47.891748 14985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:47.891993 14980 sys_catalog.cc:564] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:47.892292 14985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c [term 1 LEADER]: Becoming Leader. State: Replica: 467059f8f55e47c7a24a6a81d104575c, State: Running, Role: LEADER
I20260619 08:27:47.892447 14985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467059f8f55e47c7a24a6a81d104575c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43469 } }
I20260619 08:27:47.893446 14987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 467059f8f55e47c7a24a6a81d104575c. Latest consensus state: current_term: 1 leader_uuid: "467059f8f55e47c7a24a6a81d104575c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467059f8f55e47c7a24a6a81d104575c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43469 } } }
I20260619 08:27:47.893788 14987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:47.893596 14986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "467059f8f55e47c7a24a6a81d104575c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "467059f8f55e47c7a24a6a81d104575c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43469 } } }
I20260619 08:27:47.893990 14986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c [sys.catalog]: This master's current role is: LEADER
W20260619 08:27:47.893996 14999 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:27:47.894059 14999 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:27:47.894198 15001 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:47.894363 15001 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:47.895813 14983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:47.884938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:38859 (local address 127.0.0.1:43469)
0619 08:27:47.885110 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:47.885121 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:47.885154 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:27:47.885583 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:47.885589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:47.885638 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:47.885771 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:47.886633 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:47.887707 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:47.888636 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:47.889063 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:47.889205 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:47.891693 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:47.891697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:47.891698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:47.891711 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:47.894062 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:47.894594 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:47.894596 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:47.894596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:47.894712 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:47.894959 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:47.894959 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:47.894960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:47.895208 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:47.895385 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:47.895474 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 08:27:47.895579 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260619 08:27:47.896545 15001 catalog_manager.cc:1347] Generated new cluster ID: faf06cd4427c48c189d9daee005557db
I20260619 08:27:47.896605 15001 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:47.896791 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:27:47.907052 15001 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:47.907697 15001 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:47.908244 14983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:47.898249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:48759 (local address 127.0.0.1:43469)
0619 08:27:47.898285 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:47.898286 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:47.900367 (+ 2081us) server_negotiation.cc:407] Connection header received
0619 08:27:47.900473 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:47.900475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:47.900487 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:47.900560 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:47.901217 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:47.902007 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:47.903007 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:47.903367 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:47.903451 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:47.904964 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:47.904966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:47.904967 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:47.904975 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:47.906752 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:47.907231 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:47.907232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:47.907232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:47.907299 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:47.907543 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:47.907544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:47.907544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:47.907707 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:47.907823 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:47.908056 (+ 233us) server_negotiation.cc:299] Negotiation successful
0619 08:27:47.908124 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 08:27:47.914112 15001 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 467059f8f55e47c7a24a6a81d104575c: Generated new TSK 0
I20260619 08:27:47.914348 15001 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:47.916791 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 14912
W20260619 08:27:47.921185 14897 connection.cc:535] client connection to 127.0.0.1:43469 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43469): Connection reset by peer (error 104)
2026-06-19T08:27:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:47 dist-test-slave-5rhg krb5kdc[15011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:47 dist-test-slave-5rhg krb5kdc[15011](info): set up 2 sockets
Jun 19 08:27:47 dist-test-slave-5rhg krb5kdc[15011](info): commencing operation
krb5kdc: starting...
W20260619 08:27:48.953155 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:49 dist-test-slave-5rhg krb5kdc[15011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:27:49.070410 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45107
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:49.151464 15024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:49.151700 15024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:49.151724 15024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:49.152979 15024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:49.153023 15024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:49.153038 15024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:49.153050 15024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:49.153102 15024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:49.154606 15024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45107
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:49.154881 15024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:49.155118 15024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:49.159925 15032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:49.160072 15024 server_base.cc:1034] running on GCE node
W20260619 08:27:49.159958 15029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:49.159968 15030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:49.160388 15024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:49.160693 15024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:49.161856 15024 hybrid_clock.cc:648] HybridClock initialized: now 1781857669161849 us; error 36 us; skew 500 ppm
Jun 19 08:27:49 dist-test-slave-5rhg krb5kdc[15011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857669, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:49.164306 15024 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:27:49.164755 15024 webserver.cc:460] Webserver started at http://127.0.0.1:43913/ using document root <none> and password file <none>
I20260619 08:27:49.164989 15024 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:49.165035 15024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:49.165195 15024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:49.166272 15024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "cacff38b9e714a1fb5f6f997148c06b7"
format_stamp: "Formatted at 2026-06-19 08:27:49 on dist-test-slave-5rhg"
server_key: "8b9ff4f5bf840b19853c3f99e70c61da"
server_key_iv: "c33d4ef60ac43943d34247ec5ac6d253"
server_key_version: "encryptionkey@0"
I20260619 08:27:49.166637 15024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "cacff38b9e714a1fb5f6f997148c06b7"
format_stamp: "Formatted at 2026-06-19 08:27:49 on dist-test-slave-5rhg"
server_key: "8b9ff4f5bf840b19853c3f99e70c61da"
server_key_iv: "c33d4ef60ac43943d34247ec5ac6d253"
server_key_version: "encryptionkey@0"
I20260619 08:27:49.167938 15024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:27:49.168709 15039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:49.168879 15024 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:49.168968 15024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "cacff38b9e714a1fb5f6f997148c06b7"
format_stamp: "Formatted at 2026-06-19 08:27:49 on dist-test-slave-5rhg"
server_key: "8b9ff4f5bf840b19853c3f99e70c61da"
server_key_iv: "c33d4ef60ac43943d34247ec5ac6d253"
server_key_version: "encryptionkey@0"
I20260619 08:27:49.169040 15024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:49.197629 15024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:49.199530 15024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:49.199694 15024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:49.203959 15024 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45107
I20260619 08:27:49.204015 15091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45107 every 8 connection(s)
I20260619 08:27:49.204391 15024 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:49.205312 15092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:49.205459 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 15024
I20260619 08:27:49.205586 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:49.205821 26110 external_mini_cluster.cc:1428] Setting key a1b5dedf95ae2133af1615b3cd264bf0
I20260619 08:27:49.208665 15092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7: Bootstrap starting.
I20260619 08:27:49.209380 15092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:49.209662 15092 log.cc:824] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:49.210391 15092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7: No bootstrap required, opened a new log
I20260619 08:27:49.211413 15095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:49.206371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:50785 (local address 127.0.0.1:45107)
0619 08:27:49.206539 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:49.206546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:49.206572 (+ 26us) server_negotiation.cc:407] Connection header received
0619 08:27:49.207053 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:49.207061 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:49.207114 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:49.207250 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:49.207492 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:49.208419 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:49.209478 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:49.209922 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:49.210010 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:49.210715 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:49.210719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:49.210719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:49.210924 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:49.210965 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:49.210979 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:49.211067 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:49.211142 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 08:27:49.211169 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260619 08:27:49.212373 15092 raft_consensus.cc:348] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacff38b9e714a1fb5f6f997148c06b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } }
I20260619 08:27:49.212570 15092 raft_consensus.cc:374] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:49.212620 15092 raft_consensus.cc:729] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cacff38b9e714a1fb5f6f997148c06b7, State: Initialized, Role: FOLLOWER
I20260619 08:27:49.212749 15092 consensus_queue.cc:260] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacff38b9e714a1fb5f6f997148c06b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } }
I20260619 08:27:49.212831 15092 raft_consensus.cc:388] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:49.212870 15092 raft_consensus.cc:482] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:49.212918 15092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:49.212947 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:27:49.213450 15092 raft_consensus.cc:504] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacff38b9e714a1fb5f6f997148c06b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } }
I20260619 08:27:49.213557 15092 leader_election.cc:302] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cacff38b9e714a1fb5f6f997148c06b7; no voters:
I20260619 08:27:49.213733 15092 leader_election.cc:288] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:49.213791 15097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:49.213943 15097 raft_consensus.cc:686] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7 [term 1 LEADER]: Becoming Leader. State: Replica: cacff38b9e714a1fb5f6f997148c06b7, State: Running, Role: LEADER
I20260619 08:27:49.213987 15092 sys_catalog.cc:564] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:49.214046 15097 consensus_queue.cc:237] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacff38b9e714a1fb5f6f997148c06b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } }
I20260619 08:27:49.214468 15097 sys_catalog.cc:455] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cacff38b9e714a1fb5f6f997148c06b7. Latest consensus state: current_term: 1 leader_uuid: "cacff38b9e714a1fb5f6f997148c06b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacff38b9e714a1fb5f6f997148c06b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } } }
I20260619 08:27:49.214470 15098 sys_catalog.cc:455] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cacff38b9e714a1fb5f6f997148c06b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacff38b9e714a1fb5f6f997148c06b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } } }
I20260619 08:27:49.214577 15097 sys_catalog.cc:458] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:49.214600 15098 sys_catalog.cc:458] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:49.215476 15110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:49.216411 15110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:49.217900 15110 catalog_manager.cc:1347] Generated new cluster ID: 67928aaf76a14cadad4e3939e25b14fe
I20260619 08:27:49.217947 15110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:49.218125 15095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:49.214382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:56981 (local address 127.0.0.1:45107)
0619 08:27:49.214420 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:49.214422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:49.214638 (+ 216us) server_negotiation.cc:407] Connection header received
0619 08:27:49.214750 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:49.214750 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:49.214764 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:49.214818 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:49.215026 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:49.215687 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:49.216670 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:49.216990 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:49.217067 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:49.217781 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:49.217782 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:49.217782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:49.217787 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:49.217803 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:49.217809 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:49.217870 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:49.218035 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 08:27:49.218058 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 08:27:49.234009 15110 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:49.234812 15110 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:49.240017 15110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cacff38b9e714a1fb5f6f997148c06b7: Generated new TSK 0
I20260619 08:27:49.240248 15110 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:49.247911 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 15024
W20260619 08:27:49.251596 15009 connection.cc:535] client connection to 127.0.0.1:45107 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45107): Connection reset by peer (error 104)
2026-06-19T08:27:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1329 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:49 dist-test-slave-5rhg krb5kdc[15123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:49 dist-test-slave-5rhg krb5kdc[15123](info): set up 2 sockets
Jun 19 08:27:49 dist-test-slave-5rhg krb5kdc[15123](info): commencing operation
krb5kdc: starting...
W20260619 08:27:50.293218 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:50 dist-test-slave-5rhg krb5kdc[15123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:27:50.406389 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45913
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:50.486981 15136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:50.487183 15136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:50.487205 15136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:50.488451 15136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:50.488492 15136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:50.488507 15136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:50.488548 15136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:50.488574 15136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:50.490051 15136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45913
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:50.490301 15136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:50.490525 15136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:50.495362 15144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:50.495388 15142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:50.495404 15141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:50.495481 15136 server_base.cc:1034] running on GCE node
I20260619 08:27:50.495848 15136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:50.496089 15136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:50.497277 15136 hybrid_clock.cc:648] HybridClock initialized: now 1781857670497275 us; error 55 us; skew 500 ppm
Jun 19 08:27:50 dist-test-slave-5rhg krb5kdc[15123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857670, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:50.499718 15136 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:27:50.500097 15136 webserver.cc:460] Webserver started at http://127.0.0.1:39635/ using document root <none> and password file <none>
I20260619 08:27:50.500341 15136 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:50.500387 15136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:50.500476 15136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:50.501554 15136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "0898646914fb4feba2dab64959493bca"
format_stamp: "Formatted at 2026-06-19 08:27:50 on dist-test-slave-5rhg"
server_key: "429730c556e7c7b2a3a5b8e1f38599dd"
server_key_iv: "5e2cd35a9b97d7689c5205044ce679f6"
server_key_version: "encryptionkey@0"
I20260619 08:27:50.501852 15136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "0898646914fb4feba2dab64959493bca"
format_stamp: "Formatted at 2026-06-19 08:27:50 on dist-test-slave-5rhg"
server_key: "429730c556e7c7b2a3a5b8e1f38599dd"
server_key_iv: "5e2cd35a9b97d7689c5205044ce679f6"
server_key_version: "encryptionkey@0"
I20260619 08:27:50.503103 15136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:27:50.503755 15151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:50.503952 15136 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:27:50.504063 15136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "0898646914fb4feba2dab64959493bca"
format_stamp: "Formatted at 2026-06-19 08:27:50 on dist-test-slave-5rhg"
server_key: "429730c556e7c7b2a3a5b8e1f38599dd"
server_key_iv: "5e2cd35a9b97d7689c5205044ce679f6"
server_key_version: "encryptionkey@0"
I20260619 08:27:50.504136 15136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:50.507337 15136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:50.509373 15136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:50.509517 15136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:50.513330 15136 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45913
I20260619 08:27:50.513378 15203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45913 every 8 connection(s)
I20260619 08:27:50.513688 15136 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:50.514674 15204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:50.516984 15204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca: Bootstrap starting.
I20260619 08:27:50.517565 15204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:50.517843 15204 log.cc:824] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:50.518525 15204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca: No bootstrap required, opened a new log
I20260619 08:27:50.520093 15204 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0898646914fb4feba2dab64959493bca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45913 } }
I20260619 08:27:50.520311 15204 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:50.520363 15204 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0898646914fb4feba2dab64959493bca, State: Initialized, Role: FOLLOWER
I20260619 08:27:50.520524 15204 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0898646914fb4feba2dab64959493bca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45913 } }
I20260619 08:27:50.520617 15204 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:50.520660 15204 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:50.520713 15204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:50.521215 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 15136
I20260619 08:27:50.521241 15204 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0898646914fb4feba2dab64959493bca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45913 } }
I20260619 08:27:50.521335 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:50.521370 15204 leader_election.cc:302] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0898646914fb4feba2dab64959493bca; no voters:
I20260619 08:27:50.521534 26110 external_mini_cluster.cc:1428] Setting key 68bd1aef7ccded98898f92cbd9afb3f7
I20260619 08:27:50.521560 15204 leader_election.cc:288] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:50.521582 15207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:50.521777 15207 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca [term 1 LEADER]: Becoming Leader. State: Replica: 0898646914fb4feba2dab64959493bca, State: Running, Role: LEADER
I20260619 08:27:50.521840 15204 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:50.521919 15207 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0898646914fb4feba2dab64959493bca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45913 } }
I20260619 08:27:50.522305 15208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0898646914fb4feba2dab64959493bca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0898646914fb4feba2dab64959493bca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45913 } } }
I20260619 08:27:50.522375 15209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0898646914fb4feba2dab64959493bca. Latest consensus state: current_term: 1 leader_uuid: "0898646914fb4feba2dab64959493bca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0898646914fb4feba2dab64959493bca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45913 } } }
I20260619 08:27:50.522393 15208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:50.522485 15209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:50.522816 15218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:50.523384 15218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:50.524744 15217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:50.522502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:58189 (local address 127.0.0.1:45913)
0619 08:27:50.523085 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:50.523097 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:50.523128 (+ 31us) server_negotiation.cc:407] Connection header received
0619 08:27:50.523242 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:50.523249 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:50.523281 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:50.523419 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:50.524140 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:50.524164 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:50.524164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:50.524169 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:50.524227 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:50.524242 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:50.524298 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:50.524515 (+ 217us) server_negotiation.cc:299] Negotiation successful
0619 08:27:50.524560 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":97,"threads_started":1}
I20260619 08:27:50.525321 15218 catalog_manager.cc:1347] Generated new cluster ID: 8e4b3cd9dfdd405aa0251cb10bd52948
I20260619 08:27:50.525370 15218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:50.525688 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:27:50.528235 15217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:50.527103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:57899 (local address 127.0.0.1:45913)
0619 08:27:50.527144 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:50.527146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:50.527311 (+ 165us) server_negotiation.cc:407] Connection header received
0619 08:27:50.527355 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:50.527356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:50.527361 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:50.527424 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:50.527994 (+ 570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:50.527995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:50.527995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:50.527999 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:50.528014 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:50.528018 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:50.528073 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:50.528133 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 08:27:50.528166 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:27:50.533306 15218 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:50.533486 15218 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:50.537698 15218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0898646914fb4feba2dab64959493bca: Generated new TSK 0
I20260619 08:27:50.537889 15218 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:50.544559 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 15136
2026-06-19T08:27:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:50 dist-test-slave-5rhg krb5kdc[15235](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:50 dist-test-slave-5rhg krb5kdc[15235](info): set up 2 sockets
Jun 19 08:27:50 dist-test-slave-5rhg krb5kdc[15235](info): commencing operation
krb5kdc: starting...
W20260619 08:27:51.577082 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:51 dist-test-slave-5rhg krb5kdc[15235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:27:51.688669 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:33619
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:46875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:33619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:51.768816 15248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:51.769053 15248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:51.769093 15248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:51.770236 15248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:51.770303 15248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:51.770328 15248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:51.770356 15248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:51.770386 15248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:51.771827 15248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:46875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:33619
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:33619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:51.772094 15248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:51.772327 15248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:51.776911 15253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:51.776917 15254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:51.776959 15256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:51.777204 15248 server_base.cc:1034] running on GCE node
I20260619 08:27:51.777441 15248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:51.777783 15248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:51.778934 15248 hybrid_clock.cc:648] HybridClock initialized: now 1781857671778920 us; error 30 us; skew 500 ppm
Jun 19 08:27:51 dist-test-slave-5rhg krb5kdc[15235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857671, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:51.781396 15248 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:27:51.781862 15248 webserver.cc:460] Webserver started at http://127.25.127.190:44155/ using document root <none> and password file <none>
I20260619 08:27:51.782079 15248 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:51.782126 15248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:51.782272 15248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:51.783368 15248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "d870ed9693d14b71b0b8a5bf05dd8635"
format_stamp: "Formatted at 2026-06-19 08:27:51 on dist-test-slave-5rhg"
server_key: "00561abc1e08f8f7bde80059d1da5259"
server_key_iv: "bb037819a3adaff03377c66c8944fbf8"
server_key_version: "encryptionkey@0"
I20260619 08:27:51.783725 15248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "d870ed9693d14b71b0b8a5bf05dd8635"
format_stamp: "Formatted at 2026-06-19 08:27:51 on dist-test-slave-5rhg"
server_key: "00561abc1e08f8f7bde80059d1da5259"
server_key_iv: "bb037819a3adaff03377c66c8944fbf8"
server_key_version: "encryptionkey@0"
I20260619 08:27:51.785094 15248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:27:51.785777 15263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:51.785941 15248 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:51.786041 15248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "d870ed9693d14b71b0b8a5bf05dd8635"
format_stamp: "Formatted at 2026-06-19 08:27:51 on dist-test-slave-5rhg"
server_key: "00561abc1e08f8f7bde80059d1da5259"
server_key_iv: "bb037819a3adaff03377c66c8944fbf8"
server_key_version: "encryptionkey@0"
I20260619 08:27:51.786114 15248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:51.794047 15248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:51.795831 15248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:51.795979 15248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:51.799932 15248 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:33619
I20260619 08:27:51.799957 15315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:33619 every 8 connection(s)
I20260619 08:27:51.800472 15248 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:51.801313 15316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:51.802958 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 15248
I20260619 08:27:51.803104 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:51.803297 26110 external_mini_cluster.cc:1428] Setting key 2a7c30963422d2dd97c22a73fbf07873
I20260619 08:27:51.803866 15316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635: Bootstrap starting.
I20260619 08:27:51.804520 15316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:51.804829 15316 log.cc:824] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:51.805554 15316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635: No bootstrap required, opened a new log
I20260619 08:27:51.807363 15316 raft_consensus.cc:348] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d870ed9693d14b71b0b8a5bf05dd8635" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33619 } }
I20260619 08:27:51.807528 15316 raft_consensus.cc:374] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:51.807571 15316 raft_consensus.cc:729] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d870ed9693d14b71b0b8a5bf05dd8635, State: Initialized, Role: FOLLOWER
I20260619 08:27:51.807731 15316 consensus_queue.cc:260] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d870ed9693d14b71b0b8a5bf05dd8635" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33619 } }
I20260619 08:27:51.807812 15316 raft_consensus.cc:388] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:51.807857 15316 raft_consensus.cc:482] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:51.807925 15316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:51.808550 15316 raft_consensus.cc:504] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d870ed9693d14b71b0b8a5bf05dd8635" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33619 } }
I20260619 08:27:51.808677 15316 leader_election.cc:302] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d870ed9693d14b71b0b8a5bf05dd8635; no voters:
I20260619 08:27:51.808836 15316 leader_election.cc:288] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:51.808889 15321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:51.809165 15316 sys_catalog.cc:564] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:51.809181 15321 raft_consensus.cc:686] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635 [term 1 LEADER]: Becoming Leader. State: Replica: d870ed9693d14b71b0b8a5bf05dd8635, State: Running, Role: LEADER
I20260619 08:27:51.809384 15321 consensus_queue.cc:237] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d870ed9693d14b71b0b8a5bf05dd8635" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33619 } }
I20260619 08:27:51.809706 15323 sys_catalog.cc:455] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d870ed9693d14b71b0b8a5bf05dd8635" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d870ed9693d14b71b0b8a5bf05dd8635" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33619 } } }
I20260619 08:27:51.809773 15322 sys_catalog.cc:455] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d870ed9693d14b71b0b8a5bf05dd8635. Latest consensus state: current_term: 1 leader_uuid: "d870ed9693d14b71b0b8a5bf05dd8635" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d870ed9693d14b71b0b8a5bf05dd8635" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 33619 } } }
I20260619 08:27:51.809823 15323 sys_catalog.cc:458] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:51.809890 15322 sys_catalog.cc:458] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:51.810190 15330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 19 08:27:51 dist-test-slave-5rhg krb5kdc[15235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:27:51.810701 15330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:51.812754 15330 catalog_manager.cc:1347] Generated new cluster ID: 0bd50d4a1d86466d9be7e731f200ae66
I20260619 08:27:51.812812 15330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:51.814962 15319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:51.803822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38929 (local address 127.25.127.190:33619)
0619 08:27:51.804075 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:51.804085 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:51.804115 (+ 30us) server_negotiation.cc:407] Connection header received
0619 08:27:51.804649 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:51.804658 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:51.804738 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:51.804901 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:51.805691 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:51.806606 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:51.807403 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:51.808033 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:51.808192 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:51.811371 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:51.811376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:51.811376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:51.811388 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:51.813392 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:51.813795 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:51.813803 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:51.813803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:51.813884 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:51.814120 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:51.814121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:51.814121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:51.814353 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:51.814557 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:51.814675 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 08:27:51.814786 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":85,"threads_started":1}
I20260619 08:27:51.815888 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:27:51.825778 15330 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:51.826505 15319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:51.817278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34107 (local address 127.25.127.190:33619)
0619 08:27:51.817315 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:51.817317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:51.820338 (+ 3021us) server_negotiation.cc:407] Connection header received
0619 08:27:51.820412 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:51.820413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:51.820424 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:51.820468 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:51.821014 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:51.821524 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:51.822272 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:51.822508 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:51.822590 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:51.823911 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:51.823912 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:51.823912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:51.823919 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:51.825329 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:51.825717 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:51.825718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:51.825718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:51.825777 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:51.825980 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:51.825981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:51.825981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:51.826089 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:51.826208 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:51.826331 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 08:27:51.826399 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:27:51.826540 15330 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:51.832209 15330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d870ed9693d14b71b0b8a5bf05dd8635: Generated new TSK 0
I20260619 08:27:51.832420 15330 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:51.836791 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 15248
W20260619 08:27:51.840687 15233 connection.cc:535] client connection to 127.25.127.190:33619 recv error: Network error: failed to read from TLS socket (remote: 127.25.127.190:33619): Connection reset by peer (error 104)
2026-06-19T08:27:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1291 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:51 dist-test-slave-5rhg krb5kdc[15347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:51 dist-test-slave-5rhg krb5kdc[15347](info): set up 2 sockets
Jun 19 08:27:51 dist-test-slave-5rhg krb5kdc[15347](info): commencing operation
krb5kdc: starting...
W20260619 08:27:52.877039 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:52 dist-test-slave-5rhg krb5kdc[15347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:27:52.993261 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:36413
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:36977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:36413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:53.075562 15360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:53.075771 15360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:53.075793 15360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:53.076963 15360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:53.077005 15360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:53.077020 15360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:53.077056 15360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:53.077083 15360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:53.078598 15360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:36977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:36413
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:36413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:53.078850 15360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:53.079088 15360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:53.083735 15365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:53.083767 15366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:53.083739 15368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:53.084182 15360 server_base.cc:1034] running on GCE node
I20260619 08:27:53.084424 15360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:53.084682 15360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:53.085829 15360 hybrid_clock.cc:648] HybridClock initialized: now 1781857673085818 us; error 27 us; skew 500 ppm
Jun 19 08:27:53 dist-test-slave-5rhg krb5kdc[15347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857673, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:53.091537 15360 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:27:53.092008 15360 webserver.cc:460] Webserver started at http://127.25.127.190:39987/ using document root <none> and password file <none>
I20260619 08:27:53.092273 15360 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:53.092319 15360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:53.092475 15360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:53.093545 15360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "770d3cdb2ed2495c85bf01c8231e23ee"
format_stamp: "Formatted at 2026-06-19 08:27:53 on dist-test-slave-5rhg"
server_key: "429536ad2d5e1a0f229bb9977188c8d2"
server_key_iv: "6ac5e6a9c14188b76d1d90d7681cb099"
server_key_version: "encryptionkey@0"
I20260619 08:27:53.093899 15360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "770d3cdb2ed2495c85bf01c8231e23ee"
format_stamp: "Formatted at 2026-06-19 08:27:53 on dist-test-slave-5rhg"
server_key: "429536ad2d5e1a0f229bb9977188c8d2"
server_key_iv: "6ac5e6a9c14188b76d1d90d7681cb099"
server_key_version: "encryptionkey@0"
I20260619 08:27:53.095247 15360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:27:53.096009 15375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:53.096220 15360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:53.096318 15360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "770d3cdb2ed2495c85bf01c8231e23ee"
format_stamp: "Formatted at 2026-06-19 08:27:53 on dist-test-slave-5rhg"
server_key: "429536ad2d5e1a0f229bb9977188c8d2"
server_key_iv: "6ac5e6a9c14188b76d1d90d7681cb099"
server_key_version: "encryptionkey@0"
I20260619 08:27:53.096400 15360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:53.112231 15360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:53.114580 15360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:53.114754 15360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:53.118800 15360 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:36413
I20260619 08:27:53.118835 15427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:36413 every 8 connection(s)
I20260619 08:27:53.119197 15360 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:53.120072 15428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:53.122424 15428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee: Bootstrap starting.
I20260619 08:27:53.123008 15428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:53.123291 15428 log.cc:824] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:53.123965 15428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee: No bootstrap required, opened a new log
I20260619 08:27:53.125598 15428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "770d3cdb2ed2495c85bf01c8231e23ee" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36413 } }
I20260619 08:27:53.125735 15428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:53.125779 15428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770d3cdb2ed2495c85bf01c8231e23ee, State: Initialized, Role: FOLLOWER
I20260619 08:27:53.125947 15428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "770d3cdb2ed2495c85bf01c8231e23ee" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36413 } }
I20260619 08:27:53.126032 15428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:53.126082 15428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:53.126133 15428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:53.126976 15428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "770d3cdb2ed2495c85bf01c8231e23ee" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36413 } }
I20260619 08:27:53.127091 15428 leader_election.cc:302] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 770d3cdb2ed2495c85bf01c8231e23ee; no voters:
I20260619 08:27:53.127270 15428 leader_election.cc:288] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:53.127305 15431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:53.127466 15431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee [term 1 LEADER]: Becoming Leader. State: Replica: 770d3cdb2ed2495c85bf01c8231e23ee, State: Running, Role: LEADER
I20260619 08:27:53.127527 15428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:53.127606 15431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "770d3cdb2ed2495c85bf01c8231e23ee" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36413 } }
I20260619 08:27:53.128026 15432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "770d3cdb2ed2495c85bf01c8231e23ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "770d3cdb2ed2495c85bf01c8231e23ee" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36413 } } }
I20260619 08:27:53.128047 15433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 770d3cdb2ed2495c85bf01c8231e23ee. Latest consensus state: current_term: 1 leader_uuid: "770d3cdb2ed2495c85bf01c8231e23ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "770d3cdb2ed2495c85bf01c8231e23ee" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 36413 } } }
I20260619 08:27:53.128121 15432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:53.128129 15433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:53.128342 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 15360
I20260619 08:27:53.128453 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:53.128657 26110 external_mini_cluster.cc:1428] Setting key 68bf1c870774302508b193bd5ba2e2f8
I20260619 08:27:53.129273 15440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:53.129812 15440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:53.131623 15440 catalog_manager.cc:1347] Generated new cluster ID: 7e21a867214e48ce85a7cab0ef38f2b7
I20260619 08:27:53.131673 15440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:53.134963 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:53.129143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57041 (local address 127.25.127.190:36413)
0619 08:27:53.129540 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:53.129551 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:53.129584 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:27:53.129710 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:53.129717 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:53.129774 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:53.130181 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:53.130271 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:53.131490 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:53.132705 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:53.133309 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:53.133407 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:53.134194 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:53.134198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:53.134198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:53.134396 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:53.134432 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:53.134446 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:53.134521 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:53.134630 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 08:27:53.134753 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":65,"threads_started":1}
I20260619 08:27:53.135910 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:27:53.141110 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:53.137527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56609 (local address 127.25.127.190:36413)
0619 08:27:53.137571 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:53.137572 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:53.137605 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:27:53.137664 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:53.137665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:53.137675 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:53.137740 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:53.137859 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:53.138596 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:53.139565 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:53.139857 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:53.139944 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:53.140800 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:53.140801 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:53.140802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:53.140806 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:53.140822 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:53.140827 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:53.140902 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:53.141018 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 08:27:53.141031 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:27:53.141772 15440 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:53.142458 15440 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:53.151654 15440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 770d3cdb2ed2495c85bf01c8231e23ee: Generated new TSK 0
I20260619 08:27:53.151870 15440 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:53.158982 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 15360
W20260619 08:27:53.162997 15345 connection.cc:535] client connection to 127.25.127.190:36413 recv error: Network error: failed to read from TLS socket (remote: 127.25.127.190:36413): Connection reset by peer (error 104)
2026-06-19T08:27:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1322 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:53 dist-test-slave-5rhg krb5kdc[15459](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:53 dist-test-slave-5rhg krb5kdc[15459](info): set up 2 sockets
Jun 19 08:27:53 dist-test-slave-5rhg krb5kdc[15459](info): commencing operation
krb5kdc: starting...
W20260619 08:27:54.205020 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:54 dist-test-slave-5rhg krb5kdc[15459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:27:54.320773 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:40305
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:34019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:40305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:54.401254 15472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:54.401495 15472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:54.401535 15472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:54.402726 15472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:54.402791 15472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:54.402814 15472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:54.402848 15472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:54.402876 15472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:54.404374 15472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:34019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:40305
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:40305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:54.404656 15472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:54.404884 15472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:54.409961 15477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:54.410039 15480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:54.409978 15478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:54.410301 15472 server_base.cc:1034] running on GCE node
I20260619 08:27:54.410583 15472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:54.410912 15472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:54.412118 15472 hybrid_clock.cc:648] HybridClock initialized: now 1781857674412083 us; error 39 us; skew 500 ppm
Jun 19 08:27:54 dist-test-slave-5rhg krb5kdc[15459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857674, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:54.414682 15472 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:27:54.415277 15472 webserver.cc:460] Webserver started at http://127.25.127.190:38969/ using document root <none> and password file <none>
I20260619 08:27:54.415542 15472 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:54.415616 15472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:54.415766 15472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:54.417055 15472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "c7a1de2f732c4123bcfd643dfaaae718"
format_stamp: "Formatted at 2026-06-19 08:27:54 on dist-test-slave-5rhg"
server_key: "1d35fd5f448535d26af73a952dc04ecf"
server_key_iv: "500be74ed7f22a21663b4fb0a7ebe95b"
server_key_version: "encryptionkey@0"
I20260619 08:27:54.417451 15472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "c7a1de2f732c4123bcfd643dfaaae718"
format_stamp: "Formatted at 2026-06-19 08:27:54 on dist-test-slave-5rhg"
server_key: "1d35fd5f448535d26af73a952dc04ecf"
server_key_iv: "500be74ed7f22a21663b4fb0a7ebe95b"
server_key_version: "encryptionkey@0"
I20260619 08:27:54.418954 15472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:27:54.419835 15487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:54.420056 15472 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:54.420205 15472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "c7a1de2f732c4123bcfd643dfaaae718"
format_stamp: "Formatted at 2026-06-19 08:27:54 on dist-test-slave-5rhg"
server_key: "1d35fd5f448535d26af73a952dc04ecf"
server_key_iv: "500be74ed7f22a21663b4fb0a7ebe95b"
server_key_version: "encryptionkey@0"
I20260619 08:27:54.420300 15472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:54.423908 15472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:54.425998 15472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:54.426180 15472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:54.430476 15472 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:40305
I20260619 08:27:54.430505 15539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:40305 every 8 connection(s)
I20260619 08:27:54.430861 15472 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:54.431903 15540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:54.434258 15540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718: Bootstrap starting.
I20260619 08:27:54.434870 15540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:54.435153 15540 log.cc:824] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:54.435420 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 15472
I20260619 08:27:54.435547 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:54.435766 26110 external_mini_cluster.cc:1428] Setting key 371fd7756eaf1ff840dd10bf07ea64e5
I20260619 08:27:54.435981 15540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718: No bootstrap required, opened a new log
I20260619 08:27:54.438340 15540 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7a1de2f732c4123bcfd643dfaaae718" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40305 } }
I20260619 08:27:54.438517 15540 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:54.438575 15540 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7a1de2f732c4123bcfd643dfaaae718, State: Initialized, Role: FOLLOWER
I20260619 08:27:54.438705 15540 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7a1de2f732c4123bcfd643dfaaae718" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40305 } }
I20260619 08:27:54.438807 15540 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:54.438871 15544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:54.436324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50345 (local address 127.25.127.190:40305)
0619 08:27:54.436521 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:54.436530 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:54.436563 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:27:54.437302 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:54.437310 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:54.437338 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:54.437456 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:54.438138 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:54.438142 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:54.438142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:54.438146 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:54.438182 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:54.438192 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:54.438246 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:54.438333 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 08:27:54.438389 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
I20260619 08:27:54.438863 15540 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:54.438982 15540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:54.439795 15540 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7a1de2f732c4123bcfd643dfaaae718" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40305 } }
I20260619 08:27:54.439829 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:27:54.439910 15540 leader_election.cc:302] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7a1de2f732c4123bcfd643dfaaae718; no voters:
I20260619 08:27:54.440061 15540 leader_election.cc:288] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:54.440109 15545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:54.440306 15545 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718 [term 1 LEADER]: Becoming Leader. State: Replica: c7a1de2f732c4123bcfd643dfaaae718, State: Running, Role: LEADER
I20260619 08:27:54.440331 15540 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:54.441838 15545 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7a1de2f732c4123bcfd643dfaaae718" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40305 } }
I20260619 08:27:54.442554 15546 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7a1de2f732c4123bcfd643dfaaae718" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7a1de2f732c4123bcfd643dfaaae718" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40305 } } }
I20260619 08:27:54.442657 15546 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:54.442523 15547 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7a1de2f732c4123bcfd643dfaaae718. Latest consensus state: current_term: 1 leader_uuid: "c7a1de2f732c4123bcfd643dfaaae718" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7a1de2f732c4123bcfd643dfaaae718" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 40305 } } }
I20260619 08:27:54.442772 15547 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718 [sys.catalog]: This master's current role is: LEADER
W20260619 08:27:54.443079 15564 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:27:54.443154 15564 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:27:54.443305 15565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:54.443307 15544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:54.441398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36549 (local address 127.25.127.190:40305)
0619 08:27:54.441434 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:54.441436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:54.441544 (+ 108us) server_negotiation.cc:407] Connection header received
0619 08:27:54.441632 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:54.441633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:54.441638 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:54.441796 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:54.442970 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:54.442971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:54.442971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:54.442978 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:54.442995 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:54.443000 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:54.443059 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:54.443233 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 08:27:54.443247 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:27:54.443573 15565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:54.445130 15565 catalog_manager.cc:1347] Generated new cluster ID: 8b72fe95b2ed4b989c277ab309c47563
I20260619 08:27:54.445192 15565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:54.452281 15565 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:54.452461 15565 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:54.465776 15565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7a1de2f732c4123bcfd643dfaaae718: Generated new TSK 0
I20260619 08:27:54.466060 15565 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:54.474193 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 15472
2026-06-19T08:27:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1315 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:54 dist-test-slave-5rhg krb5kdc[15571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:54 dist-test-slave-5rhg krb5kdc[15571](info): set up 2 sockets
Jun 19 08:27:54 dist-test-slave-5rhg krb5kdc[15571](info): commencing operation
krb5kdc: starting...
W20260619 08:27:55.505110 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:55 dist-test-slave-5rhg krb5kdc[15571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:27:55.621053 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:46861
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:36933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:46861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:55.703698 15584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:55.703934 15584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:55.703972 15584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:55.705154 15584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:55.705217 15584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:55.705240 15584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:55.705261 15584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:55.705288 15584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:55.706733 15584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:36933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:46861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:46861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:55.707002 15584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:55.707228 15584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:55.711979 15589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:55.711983 15592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:55.712038 15590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:55.712277 15584 server_base.cc:1034] running on GCE node
I20260619 08:27:55.712496 15584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:55.712745 15584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:55.713886 15584 hybrid_clock.cc:648] HybridClock initialized: now 1781857675713873 us; error 29 us; skew 500 ppm
Jun 19 08:27:55 dist-test-slave-5rhg krb5kdc[15571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857675, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:55.716356 15584 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:27:55.716799 15584 webserver.cc:460] Webserver started at http://127.25.127.190:34515/ using document root <none> and password file <none>
I20260619 08:27:55.717010 15584 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:55.717053 15584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:55.717214 15584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:55.718250 15584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "bcfadfdd22c04d6d84718e775303835e"
format_stamp: "Formatted at 2026-06-19 08:27:55 on dist-test-slave-5rhg"
server_key: "60806b8dd28388afa757affa79c0d359"
server_key_iv: "fafaaebd611e82bc9d236eca34e1a374"
server_key_version: "encryptionkey@0"
I20260619 08:27:55.718595 15584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "bcfadfdd22c04d6d84718e775303835e"
format_stamp: "Formatted at 2026-06-19 08:27:55 on dist-test-slave-5rhg"
server_key: "60806b8dd28388afa757affa79c0d359"
server_key_iv: "fafaaebd611e82bc9d236eca34e1a374"
server_key_version: "encryptionkey@0"
I20260619 08:27:55.719894 15584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:27:55.720700 15599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:55.720876 15584 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:55.720963 15584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "bcfadfdd22c04d6d84718e775303835e"
format_stamp: "Formatted at 2026-06-19 08:27:55 on dist-test-slave-5rhg"
server_key: "60806b8dd28388afa757affa79c0d359"
server_key_iv: "fafaaebd611e82bc9d236eca34e1a374"
server_key_version: "encryptionkey@0"
I20260619 08:27:55.721050 15584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:55.751114 15584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:55.753273 15584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:55.753432 15584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:55.757573 15584 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:46861
I20260619 08:27:55.757612 15651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:46861 every 8 connection(s)
I20260619 08:27:55.757969 15584 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:55.758792 15652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:55.761234 15652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e: Bootstrap starting.
I20260619 08:27:55.761842 15652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:55.762145 15652 log.cc:824] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:55.762828 15652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e: No bootstrap required, opened a new log
I20260619 08:27:55.764494 15652 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcfadfdd22c04d6d84718e775303835e" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46861 } }
I20260619 08:27:55.764662 15652 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:55.764707 15652 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcfadfdd22c04d6d84718e775303835e, State: Initialized, Role: FOLLOWER
I20260619 08:27:55.764871 15652 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcfadfdd22c04d6d84718e775303835e" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46861 } }
I20260619 08:27:55.764964 15652 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:55.765014 15652 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:55.765066 15652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:55.765969 15652 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcfadfdd22c04d6d84718e775303835e" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46861 } }
I20260619 08:27:55.766074 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 15584
I20260619 08:27:55.766103 15652 leader_election.cc:302] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcfadfdd22c04d6d84718e775303835e; no voters:
I20260619 08:27:55.766186 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:55.766306 15652 leader_election.cc:288] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:55.766367 15655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:55.766400 26110 external_mini_cluster.cc:1428] Setting key 4aaa41a7f8a9a2858d7d85d053eaf973
I20260619 08:27:55.766517 15655 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e [term 1 LEADER]: Becoming Leader. State: Replica: bcfadfdd22c04d6d84718e775303835e, State: Running, Role: LEADER
I20260619 08:27:55.766607 15652 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:55.766664 15655 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcfadfdd22c04d6d84718e775303835e" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46861 } }
I20260619 08:27:55.767066 15655 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcfadfdd22c04d6d84718e775303835e. Latest consensus state: current_term: 1 leader_uuid: "bcfadfdd22c04d6d84718e775303835e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcfadfdd22c04d6d84718e775303835e" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46861 } } }
I20260619 08:27:55.767099 15657 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcfadfdd22c04d6d84718e775303835e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcfadfdd22c04d6d84718e775303835e" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46861 } } }
I20260619 08:27:55.767215 15655 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:55.767277 15657 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:55.768242 15664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:55.768739 15664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:55.770196 15664 catalog_manager.cc:1347] Generated new cluster ID: ae9856f7404248818e35080f0e15e9f6
I20260619 08:27:55.770249 15664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:27:55 dist-test-slave-5rhg krb5kdc[15571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:27:55.780351 15664 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:55.781098 15664 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:55.783604 15666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:55.767311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36187 (local address 127.25.127.190:46861)
0619 08:27:55.769283 (+ 1972us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:55.769292 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:55.769333 (+ 41us) server_negotiation.cc:407] Connection header received
0619 08:27:55.769446 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:55.769453 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:55.769526 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:55.769683 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:55.770462 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:55.771899 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:55.773005 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:55.773690 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:55.773807 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:55.777069 (+ 3262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:55.777073 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:55.777073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:55.777097 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:55.781949 (+ 4852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:55.782391 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:55.782394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:55.782394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:55.782505 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:55.782712 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:55.782713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:55.782713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:55.782976 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:55.783128 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:55.783236 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 08:27:55.783378 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787,"thread_start_us":191,"threads_started":1}
I20260619 08:27:55.784528 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:27:55.785141 15664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcfadfdd22c04d6d84718e775303835e: Generated new TSK 0
I20260619 08:27:55.785315 15664 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:55.792457 15666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:55.786029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49147 (local address 127.25.127.190:46861)
0619 08:27:55.786069 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:55.786071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:55.786150 (+ 79us) server_negotiation.cc:407] Connection header received
0619 08:27:55.786246 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:55.786247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:55.786254 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:55.786298 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:55.786914 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:55.787396 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:55.788477 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:55.788691 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:55.788745 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:55.790120 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:55.790121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:55.790121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:55.790127 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:55.791396 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:55.791758 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:55.791759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:55.791759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:55.791822 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:55.792010 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:55.792010 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:55.792011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:55.792105 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:55.792241 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:55.792340 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 08:27:55.792398 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:27:55.794834 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 15584
2026-06-19T08:27:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:55 dist-test-slave-5rhg krb5kdc[15683](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:55 dist-test-slave-5rhg krb5kdc[15683](info): set up 2 sockets
Jun 19 08:27:55 dist-test-slave-5rhg krb5kdc[15683](info): commencing operation
krb5kdc: starting...
W20260619 08:27:56.833271 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:56 dist-test-slave-5rhg krb5kdc[15683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:27:56.950344 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:44577
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:39855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:44577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:57.031744 15696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:57.031987 15696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:57.032024 15696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:57.033236 15696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:57.033309 15696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:57.033335 15696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:57.033356 15696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:57.033387 15696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:57.034873 15696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:39855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:44577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:44577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:57.035159 15696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:57.035398 15696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:57.040221 15701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:57.040228 15704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:57.040243 15702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:57.040643 15696 server_base.cc:1034] running on GCE node
I20260619 08:27:57.040895 15696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:57.041188 15696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:57.042338 15696 hybrid_clock.cc:648] HybridClock initialized: now 1781857677042322 us; error 30 us; skew 500 ppm
Jun 19 08:27:57 dist-test-slave-5rhg krb5kdc[15683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857677, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:57.044970 15696 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:27:57.045426 15696 webserver.cc:460] Webserver started at http://127.25.127.190:34439/ using document root <none> and password file <none>
I20260619 08:27:57.045637 15696 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:57.045677 15696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:57.045766 15696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:57.046798 15696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "2c07ef3278584a2eaba8cbd40cbf65d2"
format_stamp: "Formatted at 2026-06-19 08:27:57 on dist-test-slave-5rhg"
server_key: "b19a93e66f8ee14ff96263ba38b4563b"
server_key_iv: "85caf1d21873262a1f76ae97dbf8db01"
server_key_version: "encryptionkey@0"
I20260619 08:27:57.047093 15696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "2c07ef3278584a2eaba8cbd40cbf65d2"
format_stamp: "Formatted at 2026-06-19 08:27:57 on dist-test-slave-5rhg"
server_key: "b19a93e66f8ee14ff96263ba38b4563b"
server_key_iv: "85caf1d21873262a1f76ae97dbf8db01"
server_key_version: "encryptionkey@0"
I20260619 08:27:57.048400 15696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:27:57.049088 15711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:57.049264 15696 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:57.049372 15696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "2c07ef3278584a2eaba8cbd40cbf65d2"
format_stamp: "Formatted at 2026-06-19 08:27:57 on dist-test-slave-5rhg"
server_key: "b19a93e66f8ee14ff96263ba38b4563b"
server_key_iv: "85caf1d21873262a1f76ae97dbf8db01"
server_key_version: "encryptionkey@0"
I20260619 08:27:57.049445 15696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:57.079084 15696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:57.081629 15696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:57.081809 15696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:57.085808 15696 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:44577
I20260619 08:27:57.085848 15763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:44577 every 8 connection(s)
I20260619 08:27:57.086237 15696 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:57.087105 15764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:57.089447 15764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2: Bootstrap starting.
I20260619 08:27:57.089982 15764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:57.090230 15764 log.cc:824] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:57.090888 15764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2: No bootstrap required, opened a new log
I20260619 08:27:57.092522 15764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c07ef3278584a2eaba8cbd40cbf65d2" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44577 } }
I20260619 08:27:57.092661 15764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:57.092684 15764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c07ef3278584a2eaba8cbd40cbf65d2, State: Initialized, Role: FOLLOWER
I20260619 08:27:57.092819 15764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c07ef3278584a2eaba8cbd40cbf65d2" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44577 } }
I20260619 08:27:57.092896 15764 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:57.092919 15764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:57.092947 15764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:57.093739 15764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c07ef3278584a2eaba8cbd40cbf65d2" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44577 } }
I20260619 08:27:57.093827 15764 leader_election.cc:302] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c07ef3278584a2eaba8cbd40cbf65d2; no voters:
I20260619 08:27:57.093946 15764 leader_election.cc:288] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:57.094005 15767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:57.094174 15767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2 [term 1 LEADER]: Becoming Leader. State: Replica: 2c07ef3278584a2eaba8cbd40cbf65d2, State: Running, Role: LEADER
I20260619 08:27:57.094259 15764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:57.094305 15767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c07ef3278584a2eaba8cbd40cbf65d2" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44577 } }
I20260619 08:27:57.094687 15768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c07ef3278584a2eaba8cbd40cbf65d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c07ef3278584a2eaba8cbd40cbf65d2" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44577 } } }
I20260619 08:27:57.094722 15769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c07ef3278584a2eaba8cbd40cbf65d2. Latest consensus state: current_term: 1 leader_uuid: "2c07ef3278584a2eaba8cbd40cbf65d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c07ef3278584a2eaba8cbd40cbf65d2" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 44577 } } }
I20260619 08:27:57.094867 15769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:57.095089 15768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2 [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:57.095259 15776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:57.095638 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 15696
I20260619 08:27:57.095746 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:57.095773 15776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:57.095925 26110 external_mini_cluster.cc:1428] Setting key 9bb0b9cc45a4cb65d3484990129e7c11
I20260619 08:27:57.097447 15776 catalog_manager.cc:1347] Generated new cluster ID: 50288fea2a5142ec9a9d74e7295ab59f
I20260619 08:27:57.097497 15776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:57.102495 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:57.096585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42863 (local address 127.25.127.190:44577)
0619 08:27:57.096775 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:57.096786 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:57.096880 (+ 94us) server_negotiation.cc:407] Connection header received
0619 08:27:57.097186 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:57.097194 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:57.097253 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:57.097685 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:57.097780 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:57.099235 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:57.100325 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:57.100975 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:57.101094 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:57.101802 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:57.101806 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:57.101807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:57.102017 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:57.102058 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:57.102076 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:57.102148 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:57.102211 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 08:27:57.102277 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
I20260619 08:27:57.103583 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:27:57.108605 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:57.105073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52509 (local address 127.25.127.190:44577)
0619 08:27:57.105233 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:57.105235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:57.105243 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:27:57.105263 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:57.105264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:57.105273 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:57.105349 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:57.105537 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:57.106247 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:57.107270 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:57.107601 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:57.107658 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:57.108358 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:57.108359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:57.108360 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:57.108365 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:57.108379 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:57.108383 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:57.108433 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:57.108504 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 08:27:57.108527 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:27:57.113401 15776 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:57.113911 15776 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:57.118173 15776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c07ef3278584a2eaba8cbd40cbf65d2: Generated new TSK 0
I20260619 08:27:57.118402 15776 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:57.124423 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 15696
2026-06-19T08:27:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1329 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:57 dist-test-slave-5rhg krb5kdc[15795](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:57 dist-test-slave-5rhg krb5kdc[15795](info): set up 2 sockets
Jun 19 08:27:57 dist-test-slave-5rhg krb5kdc[15795](info): commencing operation
krb5kdc: starting...
W20260619 08:27:58.173164 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:58 dist-test-slave-5rhg krb5kdc[15795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:27:58.288343 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:41965
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:39631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:41965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:58.369541 15808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:58.369805 15808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:58.369849 15808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:58.371029 15808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:58.371101 15808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:58.371124 15808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:58.371143 15808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:58.371173 15808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:58.372709 15808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:39631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:41965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:41965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:58.372991 15808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:58.373234 15808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:58.378182 15813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:58.378176 15816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:58.378239 15814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:58.378423 15808 server_base.cc:1034] running on GCE node
I20260619 08:27:58.378726 15808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:58.379032 15808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:58.380225 15808 hybrid_clock.cc:648] HybridClock initialized: now 1781857678380216 us; error 36 us; skew 500 ppm
Jun 19 08:27:58 dist-test-slave-5rhg krb5kdc[15795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857678, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:58.387229 15808 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:27:58.387756 15808 webserver.cc:460] Webserver started at http://127.25.127.190:38907/ using document root <none> and password file <none>
I20260619 08:27:58.388007 15808 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:58.388075 15808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:58.388239 15808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:58.389325 15808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "1320cfce489e487f9df04f9c8d721bdd"
format_stamp: "Formatted at 2026-06-19 08:27:58 on dist-test-slave-5rhg"
server_key: "813c51bc465e7f025768d0a750e3eb99"
server_key_iv: "d17da820c54ac2c7cfb9e6fded7885e9"
server_key_version: "encryptionkey@0"
I20260619 08:27:58.389673 15808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "1320cfce489e487f9df04f9c8d721bdd"
format_stamp: "Formatted at 2026-06-19 08:27:58 on dist-test-slave-5rhg"
server_key: "813c51bc465e7f025768d0a750e3eb99"
server_key_iv: "d17da820c54ac2c7cfb9e6fded7885e9"
server_key_version: "encryptionkey@0"
I20260619 08:27:58.391040 15808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:27:58.392019 15823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:58.392256 15808 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:27:58.392359 15808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "1320cfce489e487f9df04f9c8d721bdd"
format_stamp: "Formatted at 2026-06-19 08:27:58 on dist-test-slave-5rhg"
server_key: "813c51bc465e7f025768d0a750e3eb99"
server_key_iv: "d17da820c54ac2c7cfb9e6fded7885e9"
server_key_version: "encryptionkey@0"
I20260619 08:27:58.392470 15808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:58.395771 15808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:58.397897 15808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:58.398058 15808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:58.402300 15808 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:41965
I20260619 08:27:58.402338 15875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:41965 every 8 connection(s)
I20260619 08:27:58.402695 15808 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:58.403110 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 15808
I20260619 08:27:58.403242 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:58.403455 26110 external_mini_cluster.cc:1428] Setting key ab167b966c7455287d42fa8d7ac9c1b3
I20260619 08:27:58.403803 15876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:58.406320 15879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:58.404015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38739 (local address 127.25.127.190:41965)
0619 08:27:58.404237 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:58.404252 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:58.404277 (+ 25us) server_negotiation.cc:407] Connection header received
0619 08:27:58.404926 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:58.404934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:58.404961 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:58.405094 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:58.405774 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:58.405778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:58.405778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:58.405782 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:58.405827 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:58.405838 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:58.405927 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:58.406026 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 08:27:58.406096 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":105,"threads_started":1}
I20260619 08:27:58.407397 15876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd: Bootstrap starting.
I20260619 08:27:58.407538 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:27:58.408062 15876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd: Neither blocks nor log segments found. Creating new log.
I20260619 08:27:58.408433 15876 log.cc:824] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd: Log is configured to *not* fsync() on all Append() calls
I20260619 08:27:58.409284 15876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd: No bootstrap required, opened a new log
I20260619 08:27:58.410187 15879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:58.409066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37617 (local address 127.25.127.190:41965)
0619 08:27:58.409100 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:58.409102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:58.409215 (+ 113us) server_negotiation.cc:407] Connection header received
0619 08:27:58.409270 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:58.409271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:58.409276 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:58.409332 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:58.409976 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:58.409977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:58.409978 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:27:58.409982 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:58.409999 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:27:58.410004 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:58.410069 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:58.410113 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 08:27:58.410126 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 08:27:58.411469 15876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1320cfce489e487f9df04f9c8d721bdd" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41965 } }
I20260619 08:27:58.411604 15876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:27:58.411628 15876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1320cfce489e487f9df04f9c8d721bdd, State: Initialized, Role: FOLLOWER
I20260619 08:27:58.411746 15876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1320cfce489e487f9df04f9c8d721bdd" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41965 } }
I20260619 08:27:58.411834 15876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:27:58.411863 15876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:27:58.411897 15876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:27:58.412484 15876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1320cfce489e487f9df04f9c8d721bdd" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41965 } }
I20260619 08:27:58.412575 15876 leader_election.cc:302] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1320cfce489e487f9df04f9c8d721bdd; no voters:
I20260619 08:27:58.412700 15876 leader_election.cc:288] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:27:58.412781 15886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:27:58.412935 15886 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd [term 1 LEADER]: Becoming Leader. State: Replica: 1320cfce489e487f9df04f9c8d721bdd, State: Running, Role: LEADER
I20260619 08:27:58.412988 15876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:27:58.413094 15886 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1320cfce489e487f9df04f9c8d721bdd" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41965 } }
I20260619 08:27:58.413527 15887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1320cfce489e487f9df04f9c8d721bdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1320cfce489e487f9df04f9c8d721bdd" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41965 } } }
I20260619 08:27:58.413537 15888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1320cfce489e487f9df04f9c8d721bdd. Latest consensus state: current_term: 1 leader_uuid: "1320cfce489e487f9df04f9c8d721bdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1320cfce489e487f9df04f9c8d721bdd" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 41965 } } }
I20260619 08:27:58.413695 15887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:58.413714 15888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd [sys.catalog]: This master's current role is: LEADER
I20260619 08:27:58.413936 15895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:27:58.414484 15895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:27:58.415966 15895 catalog_manager.cc:1347] Generated new cluster ID: ca152502c8b64960b147279977b070d3
I20260619 08:27:58.416054 15895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:27:58.431187 15895 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:27:58.431394 15895 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:27:58.442405 15895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1320cfce489e487f9df04f9c8d721bdd: Generated new TSK 0
I20260619 08:27:58.442636 15895 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:27:58.455188 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 15808
2026-06-19T08:27:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1331 ms)
[----------] 18 tests from AuthTokenIssuingTest (23909 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:27:58 dist-test-slave-5rhg krb5kdc[15907](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:27:58 dist-test-slave-5rhg krb5kdc[15907](info): set up 2 sockets
Jun 19 08:27:58 dist-test-slave-5rhg krb5kdc[15907](info): commencing operation
krb5kdc: starting...
W20260619 08:27:59.501166 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:27:59 dist-test-slave-5rhg krb5kdc[15907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:27:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:27:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:27:59.616323 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:46351
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:46351,127.25.127.189:41839,127.25.127.188:37399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:27:59.698128 15920 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 08:27:59.698351 15920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:59.698370 15920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:59.698390 15920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:59.699589 15920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:59.699632 15920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:59.699646 15920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:59.699685 15920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:59.699715 15920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:59.701319 15920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:46351,127.25.127.189:41839,127.25.127.188:37399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:46351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:59.701579 15920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:59.701807 15920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:59.706700 15925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:59.706686 15928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:59.706714 15920 server_base.cc:1034] running on GCE node
W20260619 08:27:59.706686 15926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:59.707240 15920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:59.707548 15920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:59.708707 15920 hybrid_clock.cc:648] HybridClock initialized: now 1781857679708672 us; error 47 us; skew 500 ppm
Jun 19 08:27:59 dist-test-slave-5rhg krb5kdc[15907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857679, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:59.711297 15920 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:27:59.711793 15920 webserver.cc:460] Webserver started at http://127.25.127.190:35933/ using document root <none> and password file <none>
I20260619 08:27:59.712031 15920 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:59.712100 15920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:59.712271 15920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:59.713407 15920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "565a8615744e43f9806ae761d05561e5"
format_stamp: "Formatted at 2026-06-19 08:27:59 on dist-test-slave-5rhg"
server_key: "3ebb646a0964090acdec7aba72af5021"
server_key_iv: "1aa0fc962fd2dfb1a0df9a271f893de9"
server_key_version: "encryptionkey@0"
I20260619 08:27:59.713775 15920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "565a8615744e43f9806ae761d05561e5"
format_stamp: "Formatted at 2026-06-19 08:27:59 on dist-test-slave-5rhg"
server_key: "3ebb646a0964090acdec7aba72af5021"
server_key_iv: "1aa0fc962fd2dfb1a0df9a271f893de9"
server_key_version: "encryptionkey@0"
I20260619 08:27:59.715123 15920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:27:59.715893 15935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:59.716120 15920 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:27:59.716259 15920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "565a8615744e43f9806ae761d05561e5"
format_stamp: "Formatted at 2026-06-19 08:27:59 on dist-test-slave-5rhg"
server_key: "3ebb646a0964090acdec7aba72af5021"
server_key_iv: "1aa0fc962fd2dfb1a0df9a271f893de9"
server_key_version: "encryptionkey@0"
I20260619 08:27:59.716354 15920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:59.731894 15920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:59.734277 15920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:59.734447 15920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:59.738662 15920 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:46351
I20260619 08:27:59.738698 15987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:46351 every 8 connection(s)
I20260619 08:27:59.739054 15920 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:27:59.739904 15988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:59.741175 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 15920
I20260619 08:27:59.741289 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:27:59.741494 26110 external_mini_cluster.cc:1428] Setting key 14914e40234e2320e7c6509058857a0b
I20260619 08:27:59.741729 15988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:27:59 dist-test-slave-5rhg krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857679, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.127.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.189@KRBTEST.COM" created.
I20260619 08:27:59.753075 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.742584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46964 (local address 127.25.127.190:46351)
0619 08:27:59.742747 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:59.742757 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:59.742877 (+ 120us) server_negotiation.cc:407] Connection header received
0619 08:27:59.743660 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:59.743662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:59.743678 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:59.743766 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:59.744507 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:59.745881 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:59.746806 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:59.747350 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:59.747471 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:59.749924 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:59.749931 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:59.749931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:59.749937 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:59.751337 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:59.751774 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:59.751776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:59.751777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:59.751852 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:59.752170 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:59.752172 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:59.752172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:59.752489 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:59.752558 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:59.752794 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 08:27:59.752893 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260619 08:27:59.753196 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.742548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46351 (local address 127.0.0.1:46964)
0619 08:27:59.742724 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:59.742763 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:59.743502 (+ 739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:59.743824 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:59.743827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:59.744242 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:59.744411 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:59.744416 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:59.745996 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:59.745997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:59.746713 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:59.746715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:59.747445 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:59.747446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:59.747506 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:59.748084 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:27:59.748092 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:59.749828 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:59.751440 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:59.751448 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:59.751450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:59.751697 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:59.751965 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:59.751966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:59.751966 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:59.752061 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:59.752568 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:59.752575 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:59.752655 (+ 80us) client_negotiation.cc:769] Sending connection context
0619 08:27:59.752760 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 08:27:59.753001 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":42,"thread_start_us":81,"threads_started":1}
I20260619 08:27:59.753968 15988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:27:59.754334 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.754210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:41839 (local address 127.0.0.1:53084)
0619 08:27:59.754260 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:59.754304 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.189:41839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 08:27:59.754499 15936 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.127.189:41839: connect: Connection refused (error 111)
I20260619 08:27:59.754906 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.754820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:41839 (local address 127.0.0.1:53098)
0619 08:27:59.754867 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:59.754891 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.189:41839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 08:27:59.755038 15988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.189:41839: Network error: Client connection negotiation failed: client connection to 127.25.127.189:41839: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.189.keytab.
Entry for principal kudu/127.25.127.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.189.keytab.
I20260619 08:27:59.766275 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.189:41839
--webserver_interface=127.25.127.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.189.keytab
--principal=kudu/127.25.127.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:46351,127.25.127.189:41839,127.25.127.188:37399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:27:59.804262 15988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } attempt: 1
I20260619 08:27:59.804832 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.804673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:41839 (local address 127.0.0.1:53110)
0619 08:27:59.804743 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:59.804794 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.189:41839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 08:27:59.805240 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.805149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:41839 (local address 127.0.0.1:53126)
0619 08:27:59.805192 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:59.805220 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.189:41839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 08:27:59.805336 15988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.189:41839: Network error: Client connection negotiation failed: client connection to 127.25.127.189:41839: connect: Connection refused (error 111)
W20260619 08:27:59.848639 15995 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 08:27:59.848860 15995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:27:59.848877 15995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:27:59.848896 15995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:27:59.850153 15995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:27:59.850194 15995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:27:59.850209 15995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:27:59.850222 15995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:27:59.850234 15995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:27:59.851661 15995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:46351,127.25.127.189:41839,127.25.127.188:37399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.189.keytab
--principal=kudu/127.25.127.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.189:41839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.127.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:27:59.851855 15995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:27:59.852027 15995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:27:59.857030 16003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:27:59.857024 16001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:59.857249 15995 server_base.cc:1034] running on GCE node
W20260619 08:27:59.857084 16000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:27:59.857499 15995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:27:59.857741 15995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:27:59.858929 15995 hybrid_clock.cc:648] HybridClock initialized: now 1781857679858913 us; error 34 us; skew 500 ppm
Jun 19 08:27:59 dist-test-slave-5rhg krb5kdc[15907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857679, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:27:59.861543 15995 init.cc:377] Logged in from keytab as kudu/127.25.127.189@KRBTEST.COM (short username kudu)
I20260619 08:27:59.862010 15995 webserver.cc:460] Webserver started at http://127.25.127.189:43367/ using document root <none> and password file <none>
I20260619 08:27:59.862242 15995 fs_manager.cc:359] Metadata directory not provided
I20260619 08:27:59.862310 15995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:27:59.862459 15995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:27:59.863528 15995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-1/data/instance:
uuid: "6373033d932241e6a4e35fd23a5f1b77"
format_stamp: "Formatted at 2026-06-19 08:27:59 on dist-test-slave-5rhg"
server_key: "52e0d36acae1c6af9fca6c9d4c5d4aa1"
server_key_iv: "8d81b9bab1c135f987718f594b64cd46"
server_key_version: "encryptionkey@0"
I20260619 08:27:59.863896 15995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-1/wal/instance:
uuid: "6373033d932241e6a4e35fd23a5f1b77"
format_stamp: "Formatted at 2026-06-19 08:27:59 on dist-test-slave-5rhg"
server_key: "52e0d36acae1c6af9fca6c9d4c5d4aa1"
server_key_iv: "8d81b9bab1c135f987718f594b64cd46"
server_key_version: "encryptionkey@0"
I20260619 08:27:59.865284 15995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:27:59.865980 16010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:27:59.866166 15995 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:27:59.866263 15995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-1/wal
uuid: "6373033d932241e6a4e35fd23a5f1b77"
format_stamp: "Formatted at 2026-06-19 08:27:59 on dist-test-slave-5rhg"
server_key: "52e0d36acae1c6af9fca6c9d4c5d4aa1"
server_key_iv: "8d81b9bab1c135f987718f594b64cd46"
server_key_version: "encryptionkey@0"
I20260619 08:27:59.866348 15995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:27:59.873453 15988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } attempt: 2
I20260619 08:27:59.873859 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.873721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:41839 (local address 127.0.0.1:53136)
0619 08:27:59.873781 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:59.873823 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.189:41839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 08:27:59.874193 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.874115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:41839 (local address 127.0.0.1:53146)
0619 08:27:59.874153 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:59.874177 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.189:41839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 08:27:59.874315 15988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.189:41839: Network error: Client connection negotiation failed: client connection to 127.25.127.189:41839: connect: Connection refused (error 111)
I20260619 08:27:59.878958 15995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:27:59.880837 15995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:27:59.881011 15995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:27:59.884929 15995 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.189:41839
I20260619 08:27:59.884959 16062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.189:41839 every 8 connection(s)
I20260619 08:27:59.885314 15995 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-1/data/info.pb
I20260619 08:27:59.886162 16063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:27:59.887707 16063 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:27:59.891244 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 15995
I20260619 08:27:59.891348 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-1/wal/instance
I20260619 08:27:59.891523 26110 external_mini_cluster.cc:1428] Setting key 78caf940e0cbec85b5e046b76677608b
Jun 19 08:27:59 dist-test-slave-5rhg krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857679, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.189@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:27:59.897433 16065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.888391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46351 (local address 127.0.0.1:46976)
0619 08:27:59.888583 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:59.888618 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:59.889121 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:59.889393 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:59.889396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:59.889635 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:59.889826 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:59.889830 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:59.890509 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:59.890510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:59.891492 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:59.891496 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:59.891908 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:59.891909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:59.892076 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:59.892607 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:27:59.892617 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:59.894128 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:59.896036 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:59.896043 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:59.896044 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:59.896324 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:59.896543 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:59.896543 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:59.896544 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:59.896648 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:59.896954 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:59.896958 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:59.897071 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 08:27:59.897154 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 08:27:59.897273 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":91,"threads_started":1}
I20260619 08:27:59.897436 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.888420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46976 (local address 127.25.127.190:46351)
0619 08:27:59.888464 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:59.888466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:59.888698 (+ 232us) server_negotiation.cc:407] Connection header received
0619 08:27:59.889235 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:59.889236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:59.889251 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:59.889307 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:59.889924 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:59.890410 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:59.891600 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:59.891817 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:59.891877 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:59.894222 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:59.894224 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:59.894224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:59.894232 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:59.895937 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:59.896396 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:59.896398 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:59.896399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:59.896469 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:59.896713 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:59.896714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:59.896714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:59.896882 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:59.896971 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:59.897205 (+ 234us) server_negotiation.cc:299] Negotiation successful
0619 08:27:59.897300 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:27:59.898028 16063 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.25.127.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.188@KRBTEST.COM" created.
Jun 19 08:27:59 dist-test-slave-5rhg krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857679, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.189@KRBTEST.COM for kudu/127.25.127.189@KRBTEST.COM
I20260619 08:27:59.906055 16065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.898287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:41839 (local address 127.0.0.1:53154)
0619 08:27:59.898334 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:59.898345 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:59.898385 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:59.898719 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:59.898720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:59.898831 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:59.898895 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:59.898896 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:59.900265 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:59.900266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:59.900811 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:59.900813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:59.901368 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:59.901369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:59.901435 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:59.901796 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:27:59.901801 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:59.903248 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:59.905101 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:59.905102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:59.905102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:59.905257 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:59.905472 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:59.905472 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:59.905473 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:59.905521 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:59.905837 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:59.905838 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:59.905872 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 08:27:59.905918 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 08:27:59.905988 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 08:27:59.906244 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.898345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53154 (local address 127.25.127.189:41839)
0619 08:27:59.898518 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:59.898524 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:59.898550 (+ 26us) server_negotiation.cc:407] Connection header received
0619 08:27:59.898575 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:59.898578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:59.898594 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:59.898691 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:59.898979 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:59.900186 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:59.900888 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:59.901256 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:59.901311 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:59.903344 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:59.903351 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:59.903351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:59.903357 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:59.905023 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:59.905338 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:59.905341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:59.905342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:59.905406 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:59.905579 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:59.905580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:59.905580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:59.905774 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:59.905857 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:59.905969 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 08:27:59.906059 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
I20260619 08:27:59.906864 16063 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:27:59.907205 16065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.907085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:37399 (local address 127.0.0.1:42158)
0619 08:27:59.907134 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:59.907179 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 08:27:59.907374 16013 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.127.188:37399: connect: Connection refused (error 111)
I20260619 08:27:59.907814 16065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.907725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:37399 (local address 127.0.0.1:42164)
0619 08:27:59.907773 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:59.907798 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 08:27:59.907953 16063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.188:37399: Network error: Client connection negotiation failed: client connection to 127.25.127.188:37399: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.188.keytab.
Entry for principal kudu/127.25.127.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.188.keytab.
I20260619 08:27:59.916296 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.188:37399
--webserver_interface=127.25.127.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.188.keytab
--principal=kudu/127.25.127.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:46351,127.25.127.189:41839,127.25.127.188:37399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:27:59.957224 16063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } attempt: 1
I20260619 08:27:59.957769 16065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.957626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:37399 (local address 127.0.0.1:42166)
0619 08:27:59.957696 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:59.957738 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 08:27:59.958205 16065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.958045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:37399 (local address 127.0.0.1:42168)
0619 08:27:59.958114 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:59.958168 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 08:27:59.958351 16063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.188:37399: Network error: Client connection negotiation failed: client connection to 127.25.127.188:37399: connect: Connection refused (error 111)
I20260619 08:27:59.965483 15988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } attempt: 3
Jun 19 08:27:59 dist-test-slave-5rhg krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857679, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for kudu/127.25.127.189@KRBTEST.COM
I20260619 08:27:59.973184 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.966213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53166 (local address 127.25.127.189:41839)
0619 08:27:59.966285 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 08:27:59.966286 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:27:59.966294 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:27:59.966317 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:27:59.966318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:27:59.966327 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:27:59.966393 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:27:59.966741 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:59.967248 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:59.968011 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:27:59.968341 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:27:59.968424 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:59.970517 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:27:59.970519 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:27:59.970519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:27:59.970527 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:27:59.971912 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:59.972346 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:59.972347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:59.972348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:59.972410 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:27:59.972620 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:27:59.972621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:27:59.972621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:27:59.972742 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:27:59.972818 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:27:59.972984 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 08:27:59.973084 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 08:27:59.973184 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.965903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:41839 (local address 127.0.0.1:53166)
0619 08:27:59.965963 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:59.965974 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:27:59.966019 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:27:59.966402 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:27:59.966404 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:27:59.966585 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:27:59.966655 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:59.966657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:59.967339 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:59.967341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:59.967917 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:27:59.967919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:27:59.968436 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:27:59.968438 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:27:59.968528 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:27:59.969042 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:27:59.969047 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:27:59.970421 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:27:59.972012 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:59.972014 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:59.972014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:59.972273 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:59.972479 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:27:59.972480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:27:59.972480 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:27:59.972542 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:27:59.972824 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:27:59.972825 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:27:59.972875 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 08:27:59.972958 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 08:27:59.973077 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 08:27:59.973613 15988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:27:59.973974 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.973870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:37399 (local address 127.0.0.1:42174)
0619 08:27:59.973919 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:59.973947 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 08:27:59.974346 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:27:59.974261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:37399 (local address 127.0.0.1:42190)
0619 08:27:59.974302 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 08:27:59.974325 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 08:27:59.974465 15988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.188:37399: Network error: Client connection negotiation failed: client connection to 127.25.127.188:37399: connect: Connection refused (error 111)
W20260619 08:28:00.003394 16070 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 08:28:00.003630 16070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:28:00.003651 16070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:28:00.003705 16070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:28:00.004968 16070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:28:00.005019 16070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:28:00.005038 16070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:28:00.005092 16070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:28:00.005125 16070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:28:00.005605 15988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } attempt: 1
I20260619 08:28:00.006083 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.005919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:37399 (local address 127.0.0.1:42200)
0619 08:28:00.005990 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:00.006035 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 08:28:00.006534 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.006447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:37399 (local address 127.0.0.1:42210)
0619 08:28:00.006490 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:00.006517 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 08:28:00.006678 15988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.188:37399: Network error: Client connection negotiation failed: client connection to 127.25.127.188:37399: connect: Connection refused (error 111)
I20260619 08:28:00.006968 16070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:46351,127.25.127.189:41839,127.25.127.188:37399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.188.keytab
--principal=kudu/127.25.127.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.188:37399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.127.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:28:00.007192 16070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:28:00.007393 16070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:28:00.012552 16076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:28:00.012471 16075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:28:00.012485 16078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:28:00.012746 16070 server_base.cc:1034] running on GCE node
I20260619 08:28:00.013018 16070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:28:00.013324 16070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:28:00.014521 16070 hybrid_clock.cc:648] HybridClock initialized: now 1781857680014505 us; error 28 us; skew 500 ppm
Jun 19 08:28:00 dist-test-slave-5rhg krb5kdc[15907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857680, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:28:00.017180 16070 init.cc:377] Logged in from keytab as kudu/127.25.127.188@KRBTEST.COM (short username kudu)
I20260619 08:28:00.017721 16070 webserver.cc:460] Webserver started at http://127.25.127.188:38617/ using document root <none> and password file <none>
I20260619 08:28:00.017957 16070 fs_manager.cc:359] Metadata directory not provided
I20260619 08:28:00.018028 16070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:28:00.018185 16070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:28:00.019390 16070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-2/data/instance:
uuid: "23d9fe40c67f421fb2762e3de6bf2226"
format_stamp: "Formatted at 2026-06-19 08:28:00 on dist-test-slave-5rhg"
server_key: "d847f356f4477a65ccf710cb50efa12c"
server_key_iv: "d73de766eecab596a2252a20e39a9a06"
server_key_version: "encryptionkey@0"
I20260619 08:28:00.019753 16070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-2/wal/instance:
uuid: "23d9fe40c67f421fb2762e3de6bf2226"
format_stamp: "Formatted at 2026-06-19 08:28:00 on dist-test-slave-5rhg"
server_key: "d847f356f4477a65ccf710cb50efa12c"
server_key_iv: "d73de766eecab596a2252a20e39a9a06"
server_key_version: "encryptionkey@0"
I20260619 08:28:00.021225 16070 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:28:00.022112 16085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:28:00.022293 16070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:28:00.022411 16070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-2/wal
uuid: "23d9fe40c67f421fb2762e3de6bf2226"
format_stamp: "Formatted at 2026-06-19 08:28:00 on dist-test-slave-5rhg"
server_key: "d847f356f4477a65ccf710cb50efa12c"
server_key_iv: "d73de766eecab596a2252a20e39a9a06"
server_key_version: "encryptionkey@0"
I20260619 08:28:00.022491 16070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:28:00.026490 16063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } attempt: 2
I20260619 08:28:00.026903 16065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.026777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:37399 (local address 127.0.0.1:42222)
0619 08:28:00.026835 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:00.026874 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 08:28:00.027253 16065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.027160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:37399 (local address 127.0.0.1:42238)
0619 08:28:00.027204 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:00.027229 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 08:28:00.027345 16063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.188:37399: Network error: Client connection negotiation failed: client connection to 127.25.127.188:37399: connect: Connection refused (error 111)
I20260619 08:28:00.030422 16070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:28:00.032332 16070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:28:00.032498 16070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:28:00.036777 16070 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.188:37399
I20260619 08:28:00.036826 16137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.188:37399 every 8 connection(s)
I20260619 08:28:00.037223 16070 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-2/data/info.pb
I20260619 08:28:00.038134 16138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:28:00.039785 16138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:28:00.041349 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 16070
I20260619 08:28:00.041460 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781857412505079-26110-0/minicluster-data/master-2/wal/instance
I20260619 08:28:00.041649 26110 external_mini_cluster.cc:1428] Setting key f26dd97cde6d504fe6dd3ae17ac58b06
Jun 19 08:28:00 dist-test-slave-5rhg krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857680, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.188@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
Jun 19 08:28:00 dist-test-slave-5rhg krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.188@KRBTEST.COM
I20260619 08:28:00.050686 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.040571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46982 (local address 127.25.127.190:46351)
0619 08:28:00.040653 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.040655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.040869 (+ 214us) server_negotiation.cc:407] Connection header received
0619 08:28:00.041375 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.041377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.041390 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.041521 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.042546 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.043364 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.044667 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.044964 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.045042 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.047340 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:00.047342 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:00.047343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:00.047351 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:00.049156 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.049670 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.049672 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.049672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.049739 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.050023 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.050024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.050025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.050169 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:00.050244 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.050546 (+ 302us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.050616 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:28:00.050999 16140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.040552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46351 (local address 127.0.0.1:46982)
0619 08:28:00.040714 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:00.040757 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:28:00.041251 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:28:00.041691 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:28:00.041697 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:28:00.042113 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:28:00.042413 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:00.042419 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.043437 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.043438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:00.044571 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:00.044575 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.045051 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.045052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:00.045174 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.045720 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:28:00.045731 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:28:00.047250 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:28:00.049246 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:00.049253 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:00.049255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:00.049593 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:00.049813 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:00.049814 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:00.049814 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:00.049945 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:00.050247 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:28:00.050255 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:28:00.050408 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 08:28:00.050530 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 08:28:00.050730 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260619 08:28:00.051594 16138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:28:00.051759 16142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.042125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42244 (local address 127.25.127.188:37399)
0619 08:28:00.042313 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.042317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.042345 (+ 28us) server_negotiation.cc:407] Connection header received
0619 08:28:00.042367 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.042369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.042388 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.042470 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.043302 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.044578 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.045476 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.045930 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.046005 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.048579 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:00.048588 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:00.048589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:00.048596 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:00.050351 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.050741 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.050745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.050745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.050808 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.051039 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.051039 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.051040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.051220 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:00.051349 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.051493 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.051575 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":121,"threads_started":1}
Jun 19 08:28:00 dist-test-slave-5rhg krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857680, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.188@KRBTEST.COM for kudu/127.25.127.189@KRBTEST.COM
I20260619 08:28:00.060595 16140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.051901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:41839 (local address 127.0.0.1:53176)
0619 08:28:00.051944 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:00.051955 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:28:00.051990 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:28:00.052221 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:28:00.052222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:28:00.052460 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:28:00.052525 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:00.052527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.053230 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.053231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:00.054197 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:00.054200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.054792 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.054794 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:00.054896 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.055438 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:28:00.055444 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:28:00.057046 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:28:00.059287 (+ 2241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:00.059288 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:00.059289 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:00.059553 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:00.059819 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:00.059820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:00.059820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:00.059892 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:00.060263 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:28:00.060264 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:28:00.060313 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 08:28:00.060406 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 08:28:00.060516 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260619 08:28:00.060653 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.052025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53176 (local address 127.25.127.189:41839)
0619 08:28:00.052083 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.052085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.052094 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:28:00.052122 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.052123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.052135 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.052217 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.052610 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.053155 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.054291 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.054708 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.054778 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.057121 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:00.057122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:00.057122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:00.057131 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:00.059189 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.059647 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.059649 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.059649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.059723 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.059981 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.059982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.059982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.060114 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:00.060208 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.060461 (+ 253us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.060551 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:28:00.062191 16138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:28:00 dist-test-slave-5rhg krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:28:00.066300 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.052664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46992 (local address 127.25.127.190:46351)
0619 08:28:00.052701 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.052703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.056819 (+ 4116us) server_negotiation.cc:407] Connection header received
0619 08:28:00.056886 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.056888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.056902 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.057003 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.057810 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.058517 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.059518 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.059875 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.059947 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.063032 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:00.063034 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:00.063034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:00.063041 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:00.064541 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.064876 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.064877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.064877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.064950 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.065145 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.065146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.065146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.065268 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:00.066046 (+ 778us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.066105 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.066194 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Jun 19 08:28:00 dist-test-slave-5rhg krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857680, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.188@KRBTEST.COM for kudu/127.25.127.188@KRBTEST.COM
I20260619 08:28:00.069644 16140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.062575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:37399 (local address 127.0.0.1:42250)
0619 08:28:00.062616 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:00.062628 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:28:00.062673 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:28:00.062826 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:28:00.062827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:28:00.062990 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:28:00.063032 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:00.063034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.063960 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.063961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:00.064517 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:00.064519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.064932 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.064933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:00.064975 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.065368 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:28:00.065373 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:28:00.066749 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:28:00.068577 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:00.068577 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:00.068578 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:00.068827 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:00.069015 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:00.069016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:00.069017 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:00.069082 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:00.069348 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:28:00.069349 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:28:00.069391 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 08:28:00.069462 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 08:28:00.069580 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 08:28:00.069743 16142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.062664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42250 (local address 127.25.127.188:37399)
0619 08:28:00.062703 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.062705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.062713 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:28:00.062732 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.062733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.062745 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.062807 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.063104 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.063875 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.064595 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.064864 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.064934 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.066878 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:00.066879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:00.066880 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:00.066890 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:00.068481 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.068896 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.068897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.068897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.068951 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.069177 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.069178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.069178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.069286 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:00.069336 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.069524 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.069635 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":19}
I20260619 08:28:00.071758 16138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226: Bootstrap starting.
Jun 19 08:28:00 dist-test-slave-5rhg krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.189@KRBTEST.COM
I20260619 08:28:00.072736 16138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226: Neither blocks nor log segments found. Creating new log.
I20260619 08:28:00.073040 16138 log.cc:824] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226: Log is configured to *not* fsync() on all Append() calls
I20260619 08:28:00.073869 16138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226: No bootstrap required, opened a new log
I20260619 08:28:00.075242 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.067048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53188 (local address 127.25.127.189:41839)
0619 08:28:00.067110 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.067112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.067121 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:28:00.067142 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.067143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.067153 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.067209 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.067765 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.068562 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.069511 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.069839 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.069937 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.072506 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:00.072507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:00.072508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:00.072515 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:00.074097 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.074425 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.074427 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.074427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.074482 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.074696 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.074697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.074698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.074813 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:00.075007 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.075046 (+ 39us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.075134 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 08:28:00.075518 16138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565a8615744e43f9806ae761d05561e5" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } } peers { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } } peers { permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } }
I20260619 08:28:00.075703 16138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23d9fe40c67f421fb2762e3de6bf2226, State: Initialized, Role: FOLLOWER
I20260619 08:28:00.075793 16138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565a8615744e43f9806ae761d05561e5" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } } peers { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } } peers { permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } }
I20260619 08:28:00.076185 16138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:28:00.076135 16144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565a8615744e43f9806ae761d05561e5" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } } peers { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } } peers { permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } } }
I20260619 08:28:00.076329 16144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:28:00.076943 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260619 08:28:00.078074 16155 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:28:00.078132 16155 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:28:00.077829 15962 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "565a8615744e43f9806ae761d05561e5"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46992
I20260619 08:28:00.078539 16070 master_runner.cc:186] Error getting master registration for 127.25.127.190:46351: OK, instance_id { permanent_uuid: "565a8615744e43f9806ae761d05561e5" instance_seqno: 1781857679737524 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 08:28:00.081468 16070 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 08:28:00.081801 15988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } attempt: 2
Jun 19 08:28:00 dist-test-slave-5rhg krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857679, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for kudu/127.25.127.188@KRBTEST.COM
I20260619 08:28:00.089967 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.082056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:37399 (local address 127.0.0.1:42254)
0619 08:28:00.082110 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:00.082121 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:28:00.082179 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:28:00.082307 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:28:00.082308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:28:00.082464 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:28:00.082503 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:00.082505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.083429 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.083430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:00.084282 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:00.084285 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.084771 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.084772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:00.084838 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.085320 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:28:00.085325 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:28:00.086947 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:28:00.088808 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:00.088809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:00.088809 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:00.089045 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:00.089253 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:00.089254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:00.089254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:00.089323 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:00.089628 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:28:00.089629 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:28:00.089681 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 08:28:00.089756 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 08:28:00.089859 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 08:28:00.089967 16142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.082132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42254 (local address 127.25.127.188:37399)
0619 08:28:00.082191 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.082193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.082203 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:28:00.082226 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.082227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.082237 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.082294 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.082585 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.083339 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.084364 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.084693 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.084758 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.087056 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:00.087057 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:00.087057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:00.087064 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:00.088716 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.089126 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.089127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.089127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.089184 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.089385 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.089386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.089386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.089556 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:00.089628 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.089787 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.089870 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:28:00.091576 15988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5: Bootstrap starting.
I20260619 08:28:00.092311 15988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5: Neither blocks nor log segments found. Creating new log.
I20260619 08:28:00.092618 15988 log.cc:824] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5: Log is configured to *not* fsync() on all Append() calls
I20260619 08:28:00.093370 15988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5: No bootstrap required, opened a new log
I20260619 08:28:00.094419 15988 raft_consensus.cc:348] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565a8615744e43f9806ae761d05561e5" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } } peers { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } } peers { permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } }
I20260619 08:28:00.094559 15988 raft_consensus.cc:729] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565a8615744e43f9806ae761d05561e5, State: Initialized, Role: FOLLOWER
I20260619 08:28:00.094655 15988 consensus_queue.cc:260] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565a8615744e43f9806ae761d05561e5" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } } peers { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } } peers { permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } }
I20260619 08:28:00.094938 16158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565a8615744e43f9806ae761d05561e5" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } } peers { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } } peers { permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } } }
I20260619 08:28:00.095005 15988 sys_catalog.cc:564] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:28:00.095019 16158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:28:00.095141 15962 raft_consensus.cc:482] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 08:28:00.095257 15962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:28:00.096128 15962 raft_consensus.cc:504] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565a8615744e43f9806ae761d05561e5" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } } peers { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } } peers { permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } }
W20260619 08:28:00.097266 16169 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260619 08:28:00.097191 16112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "565a8615744e43f9806ae761d05561e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "23d9fe40c67f421fb2762e3de6bf2226"
W20260619 08:28:00.097317 16169 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:28:00.097339 16112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:28:00.098171 16112 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565a8615744e43f9806ae761d05561e5 in term 1.
I20260619 08:28:00.096531 15962 leader_election.cc:288] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [CANDIDATE]: Term 1 election: Requested vote from peers 6373033d932241e6a4e35fd23a5f1b77 (127.25.127.189:41839), 23d9fe40c67f421fb2762e3de6bf2226 (127.25.127.188:37399)
I20260619 08:28:00.098462 15938 leader_election.cc:302] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23d9fe40c67f421fb2762e3de6bf2226, 565a8615744e43f9806ae761d05561e5; no voters:
I20260619 08:28:00.098587 16158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:28:00.098738 16158 raft_consensus.cc:686] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [term 1 LEADER]: Becoming Leader. State: Replica: 565a8615744e43f9806ae761d05561e5, State: Running, Role: LEADER
I20260619 08:28:00.098819 16158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565a8615744e43f9806ae761d05561e5" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } } peers { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } } peers { permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } }
I20260619 08:28:00.099254 16158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 565a8615744e43f9806ae761d05561e5. Latest consensus state: current_term: 1 leader_uuid: "565a8615744e43f9806ae761d05561e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565a8615744e43f9806ae761d05561e5" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } } peers { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } } peers { permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } } }
I20260619 08:28:00.099364 16158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [sys.catalog]: This master's current role is: LEADER
I20260619 08:28:00.099783 16177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:28:00.100179 16177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:28:00.097970 15920 master_runner.cc:186] Error getting master registration for 127.25.127.189:41839: OK, instance_id { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" instance_seqno: 1781857679883762 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 08:28:00.100289 15920 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 08:28:00.108945 16142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.099749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42266 (local address 127.25.127.188:37399)
0619 08:28:00.099788 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.099790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.100831 (+ 1041us) server_negotiation.cc:407] Connection header received
0619 08:28:00.100928 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.100929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.100939 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.100969 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.101677 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.102415 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.103490 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.103803 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.103844 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.105507 (+ 1663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:00.105508 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:00.105508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:00.105515 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:00.107148 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.107809 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.107810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.107811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.107909 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.108227 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.108228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.108228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.108368 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:00.108448 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.108763 (+ 315us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.108846 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:28:00.111757 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.099525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47002 (local address 127.25.127.190:46351)
0619 08:28:00.099590 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.099592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.100327 (+ 735us) server_negotiation.cc:407] Connection header received
0619 08:28:00.100396 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.100397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.100409 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.100472 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.101664 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.102374 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.103993 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.104316 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.104362 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.108726 (+ 4364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:00.108727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:00.108727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:00.108734 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:00.110435 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.110852 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.110853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.110853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.110909 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.111114 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.111115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.111115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.111242 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:00.111327 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.111562 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.111656 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 08:28:00.113677 16112 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226 [term 1 FOLLOWER]: Refusing update from remote peer 565a8615744e43f9806ae761d05561e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:28:00.114002 16158 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:28:00.115688 16144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 565a8615744e43f9806ae761d05561e5. Latest consensus state: current_term: 1 leader_uuid: "565a8615744e43f9806ae761d05561e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565a8615744e43f9806ae761d05561e5" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } } peers { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } } peers { permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } } }
I20260619 08:28:00.115790 16144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:28:00.117699 16144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "565a8615744e43f9806ae761d05561e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565a8615744e43f9806ae761d05561e5" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } } peers { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } } peers { permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } } }
I20260619 08:28:00.117781 16144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:28:00.118409 16171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "565a8615744e43f9806ae761d05561e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565a8615744e43f9806ae761d05561e5" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } } peers { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } } peers { permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } } }
I20260619 08:28:00.118526 16171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [sys.catalog]: This master's current role is: LEADER
I20260619 08:28:00.118624 16063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } attempt: 3
I20260619 08:28:00.119207 16181 mvcc.cc:205] Tried to move back new op lower bound from 7298489057741836288 to 7298489057685549056. Current Snapshot: MvccSnapshot[applied={T|T < 7298489057741836288}]
I20260619 08:28:00.121347 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.100423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53190 (local address 127.25.127.189:41839)
0619 08:28:00.100479 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.100481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.109720 (+ 9239us) server_negotiation.cc:407] Connection header received
0619 08:28:00.109853 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.109854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.109877 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.109907 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.110788 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.111601 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.113511 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.113861 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.113905 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.116320 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:00.116321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:00.116321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:00.116329 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:00.118199 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.119053 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.119053 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.119054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.119113 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.119695 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.119696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.119697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.119852 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:00.119944 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.121161 (+ 1217us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.121252 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 08:28:00.122202 16182 mvcc.cc:205] Tried to move back new op lower bound from 7298489057741836288 to 7298489057685549056. Current Snapshot: MvccSnapshot[applied={T|T < 7298489057741836288}]
Jun 19 08:28:00 dist-test-slave-5rhg krb5kdc[15907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857679, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.189@KRBTEST.COM for kudu/127.25.127.188@KRBTEST.COM
I20260619 08:28:00.127305 16065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.118845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:37399 (local address 127.0.0.1:42268)
0619 08:28:00.118937 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:00.118948 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:28:00.118984 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:28:00.119118 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:28:00.119120 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:28:00.119274 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:28:00.119316 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:00.119318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.120199 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.120200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:00.121030 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:00.121033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.121732 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.121733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:00.121811 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.122215 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:28:00.122220 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:28:00.123711 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:28:00.125743 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:00.125744 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:00.125744 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:00.125969 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:00.126592 (+ 623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:00.126593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:00.126593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:00.126687 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:00.126979 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:28:00.126980 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:28:00.127031 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 08:28:00.127107 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 08:28:00.127206 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260619 08:28:00.127306 16142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.118956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42268 (local address 127.25.127.188:37399)
0619 08:28:00.118997 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.118999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.119006 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:28:00.119028 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.119028 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.119038 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.119098 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.119393 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.120111 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.121115 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.121426 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.121500 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.123800 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:00.123801 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:00.123801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:00.123808 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:00.125662 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.126040 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.126041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.126041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.126129 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:00.126775 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:00.126776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:00.126777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:00.126915 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:00.126978 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.127145 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.127219 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:28:00.127436 16177 catalog_manager.cc:1347] Generated new cluster ID: 326981303f6a4287ac22d50f74333e58
I20260619 08:28:00.127475 16177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:28:00.133891 16063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77: Bootstrap starting.
I20260619 08:28:00.134644 16063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77: Neither blocks nor log segments found. Creating new log.
I20260619 08:28:00.134946 16063 log.cc:824] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77: Log is configured to *not* fsync() on all Append() calls
I20260619 08:28:00.136410 16177 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:28:00.137188 16177 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:28:00.148825 16063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77: No bootstrap required, opened a new log
I20260619 08:28:00.150372 16177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5: Generated new TSK 0
I20260619 08:28:00.150597 16177 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:28:00.151027 16063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565a8615744e43f9806ae761d05561e5" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } } peers { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } } peers { permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } }
I20260619 08:28:00.151175 16063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6373033d932241e6a4e35fd23a5f1b77, State: Initialized, Role: FOLLOWER
I20260619 08:28:00.151252 16063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565a8615744e43f9806ae761d05561e5" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } } peers { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } } peers { permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } }
I20260619 08:28:00.151579 16187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565a8615744e43f9806ae761d05561e5" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } } peers { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } } peers { permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } } }
I20260619 08:28:00.151631 16063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:28:00.151961 16036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:28:00.152966 16036 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77 [term 1 FOLLOWER]: Refusing update from remote peer 565a8615744e43f9806ae761d05561e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:28:00.153013 16187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:28:00.153294 16037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "565a8615744e43f9806ae761d05561e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6373033d932241e6a4e35fd23a5f1b77"
I20260619 08:28:00.153309 16158 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:28:00.154114 16037 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565a8615744e43f9806ae761d05561e5 in term 1.
I20260619 08:28:00.157539 16187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 565a8615744e43f9806ae761d05561e5. Latest consensus state: current_term: 1 leader_uuid: "565a8615744e43f9806ae761d05561e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565a8615744e43f9806ae761d05561e5" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } } peers { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } } peers { permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } } }
I20260619 08:28:00.157624 16187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:28:00.157660 16187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "565a8615744e43f9806ae761d05561e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565a8615744e43f9806ae761d05561e5" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } } peers { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } } peers { permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } } }
I20260619 08:28:00.157701 16187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:28:00.161379 16204 catalog_manager.cc:1259] Loaded cluster ID: 326981303f6a4287ac22d50f74333e58
I20260619 08:28:00.161533 16204 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77: loading cluster ID for follower catalog manager: success
I20260619 08:28:00.162781 16204 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77: acquiring CA information for follower catalog manager: success
I20260619 08:28:00.163239 16204 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6373033d932241e6a4e35fd23a5f1b77: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 08:28:00.185621 15991 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:00.184562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47008 (local address 127.25.127.190:46351)
0619 08:28:00.184615 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.184617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.184822 (+ 205us) server_negotiation.cc:407] Connection header received
0619 08:28:00.184935 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.184936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.184949 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.184982 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.185537 (+ 555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47008: BlockingRecv error: recv got EOF from 127.0.0.1:47008 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260619 08:28:00.186404 16142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:00.184807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42284 (local address 127.25.127.188:37399)
0619 08:28:00.184851 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.184854 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.185854 (+ 1000us) server_negotiation.cc:407] Connection header received
0619 08:28:00.185964 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.185965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.185976 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.186006 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.186344 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42284: BlockingRecv error: recv got EOF from 127.0.0.1:42284 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 08:28:00.187404 16067 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:00.186730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53204 (local address 127.25.127.189:41839)
0619 08:28:00.186773 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.186774 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.186893 (+ 119us) server_negotiation.cc:407] Connection header received
0619 08:28:00.186985 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.186985 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.186996 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.187024 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.187344 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53204: BlockingRecv error: recv got EOF from 127.0.0.1:53204 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:28:00.192747 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.189107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53216 (local address 127.25.127.189:41839)
0619 08:28:00.189153 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.189155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.189408 (+ 253us) server_negotiation.cc:407] Connection header received
0619 08:28:00.189517 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.189518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.189565 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.189611 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:00.189822 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.190613 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.191757 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.192061 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.192120 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.192368 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:00.192499 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:00.192546 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.192648 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.192668 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 08:28:00.196235 16142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:00.195387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42290 (local address 127.25.127.188:37399)
0619 08:28:00.195429 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.195430 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.195604 (+ 174us) server_negotiation.cc:407] Connection header received
0619 08:28:00.195712 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.195713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.195731 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.195761 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.196130 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42290: BlockingRecv error: recv got EOF from 127.0.0.1:42290 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:28:00.201700 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.198484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53220 (local address 127.25.127.189:41839)
0619 08:28:00.198521 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.198523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.198834 (+ 311us) server_negotiation.cc:407] Connection header received
0619 08:28:00.198925 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.198926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.198935 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.198961 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:00.199135 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.199791 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.200883 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.201172 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.201217 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.201428 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:00.201463 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:00.201493 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.201602 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.201623 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 08:28:00.205116 16142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:00.204266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42292 (local address 127.25.127.188:37399)
0619 08:28:00.204304 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.204306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.204530 (+ 224us) server_negotiation.cc:407] Connection header received
0619 08:28:00.204635 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.204636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.204648 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.204680 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.205047 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42292: BlockingRecv error: recv got EOF from 127.0.0.1:42292 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:28:00.211498 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.208312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53230 (local address 127.25.127.189:41839)
0619 08:28:00.208349 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.208351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.208598 (+ 247us) server_negotiation.cc:407] Connection header received
0619 08:28:00.208700 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.208701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.208712 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.208741 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:00.208909 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.209604 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.210680 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.210960 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.211005 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.211227 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:00.211262 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:00.211291 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.211399 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.211419 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 08:28:00.214870 16142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:00.214126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42296 (local address 127.25.127.188:37399)
0619 08:28:00.214179 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.214181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.214308 (+ 127us) server_negotiation.cc:407] Connection header received
0619 08:28:00.214409 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.214411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.214423 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.214453 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.214818 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42296: BlockingRecv error: recv got EOF from 127.0.0.1:42296 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 08:28:00.227545 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.220207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53238 (local address 127.25.127.189:41839)
0619 08:28:00.220284 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.220286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.224383 (+ 4097us) server_negotiation.cc:407] Connection header received
0619 08:28:00.224496 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.224498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.224509 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.224539 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:00.224736 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.225532 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.226711 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.226994 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.227042 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.227270 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:00.227307 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:00.227336 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.227442 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.227463 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260619 08:28:00.230894 16142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:00.230026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42312 (local address 127.25.127.188:37399)
0619 08:28:00.230082 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.230084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.230313 (+ 229us) server_negotiation.cc:407] Connection header received
0619 08:28:00.230414 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.230415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.230426 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.230457 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.230835 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42312: BlockingRecv error: recv got EOF from 127.0.0.1:42312 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:28:00.243755 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.240320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53240 (local address 127.25.127.189:41839)
0619 08:28:00.240367 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.240369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.240637 (+ 268us) server_negotiation.cc:407] Connection header received
0619 08:28:00.240756 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.240757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.240769 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.240798 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:00.240983 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.241777 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.242887 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.243212 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.243263 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.243471 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:00.243508 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:00.243536 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.243639 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.243664 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 08:28:00.247215 16142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:00.246359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42320 (local address 127.25.127.188:37399)
0619 08:28:00.246409 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.246411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.246608 (+ 197us) server_negotiation.cc:407] Connection header received
0619 08:28:00.246705 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.246706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.246717 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.246746 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.247153 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42320: BlockingRecv error: recv got EOF from 127.0.0.1:42320 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:28:00.268283 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.264750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53248 (local address 127.25.127.189:41839)
0619 08:28:00.264798 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.264800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.265105 (+ 305us) server_negotiation.cc:407] Connection header received
0619 08:28:00.265224 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.265225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.265237 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.265269 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:00.265473 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.266231 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.267389 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.267678 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.267724 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.267932 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:00.267969 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:00.267997 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.268122 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.268196 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 08:28:00.271672 16142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:00.270874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42336 (local address 127.25.127.188:37399)
0619 08:28:00.270913 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.270915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.271136 (+ 221us) server_negotiation.cc:407] Connection header received
0619 08:28:00.271228 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.271229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.271241 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.271269 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.271611 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42336: BlockingRecv error: recv got EOF from 127.0.0.1:42336 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:28:00.307832 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.305105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53258 (local address 127.25.127.189:41839)
0619 08:28:00.305171 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.305173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.305303 (+ 130us) server_negotiation.cc:407] Connection header received
0619 08:28:00.305383 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.305384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.305394 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.305446 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:00.305612 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.306102 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.307173 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.307357 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.307402 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.307581 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:00.307609 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:00.307646 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.307728 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.307744 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260619 08:28:00.310710 16142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:00.310124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42352 (local address 127.25.127.188:37399)
0619 08:28:00.310184 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.310186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.310199 (+ 13us) server_negotiation.cc:407] Connection header received
0619 08:28:00.310264 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.310265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.310277 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.310349 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.310650 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42352: BlockingRecv error: recv got EOF from 127.0.0.1:42352 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:28:00.378626 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.375831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53268 (local address 127.25.127.189:41839)
0619 08:28:00.375901 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.375904 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.376086 (+ 182us) server_negotiation.cc:407] Connection header received
0619 08:28:00.376137 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.376138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.376168 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.376220 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:00.376397 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.376976 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.377848 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.378113 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.378194 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.378337 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:00.378375 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:00.378449 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.378512 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.378533 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260619 08:28:00.381039 16142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:00.380362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42362 (local address 127.25.127.188:37399)
0619 08:28:00.380417 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.380419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.380529 (+ 110us) server_negotiation.cc:407] Connection header received
0619 08:28:00.380558 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.380558 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.380569 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.380618 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.380958 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42362: BlockingRecv error: recv got EOF from 127.0.0.1:42362 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:28:00.512738 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.510138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53272 (local address 127.25.127.189:41839)
0619 08:28:00.510227 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.510230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.510328 (+ 98us) server_negotiation.cc:407] Connection header received
0619 08:28:00.510432 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.510433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.510444 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.510513 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:00.510665 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.511154 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.511947 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.512216 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.512300 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.512445 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:00.512491 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:00.512550 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.512620 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.512642 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 08:28:00.515058 16142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:00.514385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42366 (local address 127.25.127.188:37399)
0619 08:28:00.514480 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.514482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.514497 (+ 15us) server_negotiation.cc:407] Connection header received
0619 08:28:00.514568 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.514569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.514578 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.514630 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.514982 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42366: BlockingRecv error: recv got EOF from 127.0.0.1:42366 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:28:00.583341 16158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "565a8615744e43f9806ae761d05561e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565a8615744e43f9806ae761d05561e5" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46351 } } peers { permanent_uuid: "6373033d932241e6a4e35fd23a5f1b77" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 41839 } } peers { permanent_uuid: "23d9fe40c67f421fb2762e3de6bf2226" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 37399 } } }
I20260619 08:28:00.583513 16158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 565a8615744e43f9806ae761d05561e5 [sys.catalog]: This master's current role is: LEADER
I20260619 08:28:00.774914 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:00.772294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53276 (local address 127.25.127.189:41839)
0619 08:28:00.772391 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.772394 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.772495 (+ 101us) server_negotiation.cc:407] Connection header received
0619 08:28:00.772597 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.772598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.772609 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.772684 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:00.772834 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.773392 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.774181 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:00.774395 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:00.774484 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:00.774640 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:00.774684 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:00.774739 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:00.774800 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 08:28:00.774822 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260619 08:28:00.777437 16142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:00.776907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42372 (local address 127.25.127.188:37399)
0619 08:28:00.776971 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:00.776973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:00.777006 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:28:00.777060 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:00.777061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:00.777071 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:00.777120 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:00.777356 (+ 236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42372: BlockingRecv error: recv got EOF from 127.0.0.1:42372 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:28:01.078608 16155 catalog_manager.cc:1259] Loaded cluster ID: 326981303f6a4287ac22d50f74333e58
I20260619 08:28:01.078711 16155 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226: loading cluster ID for follower catalog manager: success
I20260619 08:28:01.079574 16155 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226: acquiring CA information for follower catalog manager: success
I20260619 08:28:01.079905 16155 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 23d9fe40c67f421fb2762e3de6bf2226: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 08:28:01.293424 16320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:01.290675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53280 (local address 127.25.127.189:41839)
0619 08:28:01.290810 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:01.290813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:01.290920 (+ 107us) server_negotiation.cc:407] Connection header received
0619 08:28:01.291027 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:01.291028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:01.291048 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:01.291124 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:01.291269 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:01.291803 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:01.292665 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:01.292888 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:01.292972 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:01.293125 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:01.293162 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:01.293227 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:01.293289 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 08:28:01.293314 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
I20260619 08:28:01.297983 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:01.295332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42382 (local address 127.25.127.188:37399)
0619 08:28:01.295470 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:01.295472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:01.295539 (+ 67us) server_negotiation.cc:407] Connection header received
0619 08:28:01.295612 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:01.295612 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:01.295659 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:01.295726 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:01.295821 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:01.296429 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:01.297160 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:01.297385 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:01.297439 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:01.297697 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:01.297811 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:01.297883 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:01.297903 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 08:28:01.297922 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":55,"threads_started":1}
I20260619 08:28:01.302584 16320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:01.299623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53288 (local address 127.25.127.189:41839)
0619 08:28:01.299672 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:01.299674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:01.299787 (+ 113us) server_negotiation.cc:407] Connection header received
0619 08:28:01.300052 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:01.300054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:01.300065 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:01.300112 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:01.300320 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:01.301039 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:01.301746 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:01.302040 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:01.302104 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:01.302253 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:01.302285 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:01.302340 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:01.302469 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 08:28:01.302497 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 08:28:01.302791 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:01.299773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42394 (local address 127.25.127.188:37399)
0619 08:28:01.299816 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:01.299817 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:01.299929 (+ 112us) server_negotiation.cc:407] Connection header received
0619 08:28:01.300002 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:01.300002 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:01.300013 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:01.300068 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:01.300233 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:01.300963 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:01.301938 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:01.302178 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:01.302246 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:01.302513 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:01.302541 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:01.302567 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:01.302676 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 08:28:01.302702 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:28:01.302943 16333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:01.299628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47024 (local address 127.25.127.190:46351)
0619 08:28:01.299765 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:01.299767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:01.299872 (+ 105us) server_negotiation.cc:407] Connection header received
0619 08:28:01.300001 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:01.300003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:01.300057 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:01.300135 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:01.300290 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:01.300804 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:01.301876 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:01.302187 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:01.302260 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:01.302510 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:01.302625 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:01.302682 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:01.302834 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 08:28:01.302861 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260619 08:28:01.304664 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 15920
I20260619 08:28:01.309923 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 15995
I20260619 08:28:01.314697 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 16070
2026-06-19T08:28:01Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2861 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:28:01 dist-test-slave-5rhg krb5kdc[16337](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:28:01 dist-test-slave-5rhg krb5kdc[16337](info): set up 2 sockets
Jun 19 08:28:01 dist-test-slave-5rhg krb5kdc[16337](info): commencing operation
krb5kdc: starting...
W20260619 08:28:02.341253 26110 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/test-user.keytab.
Jun 19 08:28:02 dist-test-slave-5rhg krb5kdc[16337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:28:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:28:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.127.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
Entry for principal kudu/127.25.127.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab.
I20260619 08:28:02.456836 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:46049
--webserver_interface=127.25.127.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:46089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:46049,127.25.127.189:37955,127.25.127.188:42267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:28:02.538055 16350 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 08:28:02.538308 16350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:28:02.538339 16350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:28:02.538385 16350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:28:02.539549 16350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:28:02.539615 16350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:28:02.539639 16350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:28:02.539661 16350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:28:02.539691 16350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:28:02.541167 16350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:46089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:46049,127.25.127.189:37955,127.25.127.188:42267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.190.keytab
--principal=kudu/127.25.127.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.190:46049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.127.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:28:02.541451 16350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:28:02.541687 16350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:28:02.546578 16355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:28:02.546581 16358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:28:02.546705 16350 server_base.cc:1034] running on GCE node
W20260619 08:28:02.546555 16356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:28:02.547070 16350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:28:02.547402 16350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:28:02.548564 16350 hybrid_clock.cc:648] HybridClock initialized: now 1781857682548552 us; error 35 us; skew 500 ppm
Jun 19 08:28:02 dist-test-slave-5rhg krb5kdc[16337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857682, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:28:02.551051 16350 init.cc:377] Logged in from keytab as kudu/127.25.127.190@KRBTEST.COM (short username kudu)
I20260619 08:28:02.551537 16350 webserver.cc:460] Webserver started at http://127.25.127.190:35845/ using document root <none> and password file <none>
I20260619 08:28:02.551760 16350 fs_manager.cc:359] Metadata directory not provided
I20260619 08:28:02.551826 16350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:28:02.551975 16350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:28:02.553103 16350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-0/data/instance:
uuid: "d3bfbfc148d14293b72ad50e72925bf3"
format_stamp: "Formatted at 2026-06-19 08:28:02 on dist-test-slave-5rhg"
server_key: "cd9a3a204c45d239a466c10fb73b1ff3"
server_key_iv: "d71587f305f43c4a049a6145a0ac771b"
server_key_version: "encryptionkey@0"
I20260619 08:28:02.553475 16350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-0/wal/instance:
uuid: "d3bfbfc148d14293b72ad50e72925bf3"
format_stamp: "Formatted at 2026-06-19 08:28:02 on dist-test-slave-5rhg"
server_key: "cd9a3a204c45d239a466c10fb73b1ff3"
server_key_iv: "d71587f305f43c4a049a6145a0ac771b"
server_key_version: "encryptionkey@0"
I20260619 08:28:02.554791 16350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:28:02.555521 16365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:28:02.555699 16350 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:28:02.555797 16350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-0/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-0/wal
uuid: "d3bfbfc148d14293b72ad50e72925bf3"
format_stamp: "Formatted at 2026-06-19 08:28:02 on dist-test-slave-5rhg"
server_key: "cd9a3a204c45d239a466c10fb73b1ff3"
server_key_iv: "d71587f305f43c4a049a6145a0ac771b"
server_key_version: "encryptionkey@0"
I20260619 08:28:02.555883 16350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:28:02.591220 16350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:28:02.593755 16350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:28:02.593926 16350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:28:02.597949 16350 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.190:46049
I20260619 08:28:02.597982 16417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.190:46049 every 8 connection(s)
I20260619 08:28:02.598352 16350 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-0/data/info.pb
I20260619 08:28:02.599176 16418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:28:02.600843 16418 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:28:02.601852 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 16350
I20260619 08:28:02.601979 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-0/wal/instance
I20260619 08:28:02.602185 26110 external_mini_cluster.cc:1428] Setting key e7b0100a666ff8138e4ceb259d1135d9
Jun 19 08:28:02 dist-test-slave-5rhg krb5kdc[16337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857682, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.127.189@KRBTEST.COM; defaulting to no policy
I20260619 08:28:02.610792 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.601629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48726 (local address 127.25.127.190:46049)
0619 08:28:02.601747 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:02.601751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:02.601787 (+ 36us) server_negotiation.cc:407] Connection header received
0619 08:28:02.602487 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:02.602489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:02.602505 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:02.602581 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:02.603322 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.604121 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.605049 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.605354 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.605468 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.607534 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:02.607543 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:02.607543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:02.607554 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:02.609175 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.609581 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.609583 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.609584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.609669 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.609901 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.609902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.609902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.610116 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:02.610208 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:02.610392 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 08:28:02.610537 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":64,"threads_started":1}
I20260619 08:28:02.610791 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.601482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46049 (local address 127.0.0.1:48726)
0619 08:28:02.601628 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.601678 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 08:28:02.602320 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:28:02.602619 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:28:02.602623 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:28:02.602986 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:28:02.603206 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:02.603211 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.604264 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.604267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:02.604949 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:02.604951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.605442 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.605443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:02.605507 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.606060 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:28:02.606069 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:28:02.607435 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:28:02.609287 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:02.609293 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:02.609296 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:02.609489 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:02.609734 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:02.609735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:02.609735 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:02.609837 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:02.610188 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:28:02.610193 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:28:02.610252 (+ 59us) client_negotiation.cc:769] Sending connection context
0619 08:28:02.610360 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 08:28:02.610600 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":100,"thread_start_us":66,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.189@KRBTEST.COM" created.
I20260619 08:28:02.611613 16418 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:28:02.611975 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.611860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:37955 (local address 127.0.0.1:45882)
0619 08:28:02.611906 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.611947 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.189:37955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 08:28:02.612140 16369 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.127.189:37955: connect: Connection refused (error 111)
I20260619 08:28:02.612599 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.612496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:37955 (local address 127.0.0.1:45888)
0619 08:28:02.612557 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.612582 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.189:37955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 08:28:02.612720 16418 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.189:37955: Network error: Client connection negotiation failed: client connection to 127.25.127.189:37955: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.189.keytab.
Entry for principal kudu/127.25.127.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.189.keytab.
I20260619 08:28:02.624392 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.189:37955
--webserver_interface=127.25.127.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.189.keytab
--principal=kudu/127.25.127.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:46089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:46049,127.25.127.189:37955,127.25.127.188:42267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:28:02.661911 16418 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } attempt: 1
I20260619 08:28:02.662441 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.662290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:37955 (local address 127.0.0.1:45898)
0619 08:28:02.662360 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.662404 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.189:37955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 08:28:02.662827 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.662730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:37955 (local address 127.0.0.1:45904)
0619 08:28:02.662769 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.662794 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.189:37955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 08:28:02.662942 16418 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.189:37955: Network error: Client connection negotiation failed: client connection to 127.25.127.189:37955: connect: Connection refused (error 111)
W20260619 08:28:02.705984 16425 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 08:28:02.706256 16425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:28:02.706296 16425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:28:02.706360 16425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:28:02.707583 16425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:28:02.707652 16425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:28:02.707675 16425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:28:02.707697 16425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:28:02.707721 16425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:28:02.709363 16425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:46089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:46049,127.25.127.189:37955,127.25.127.188:42267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.189.keytab
--principal=kudu/127.25.127.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.189:37955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.127.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:28:02.709648 16425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:28:02.709874 16425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:28:02.714847 16425 server_base.cc:1034] running on GCE node
W20260619 08:28:02.714824 16431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:28:02.714818 16430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:28:02.714819 16433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:28:02.715200 16425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:28:02.715502 16425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:28:02.716665 16425 hybrid_clock.cc:648] HybridClock initialized: now 1781857682716652 us; error 46 us; skew 500 ppm
Jun 19 08:28:02 dist-test-slave-5rhg krb5kdc[16337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857682, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:28:02.719158 16425 init.cc:377] Logged in from keytab as kudu/127.25.127.189@KRBTEST.COM (short username kudu)
I20260619 08:28:02.719638 16425 webserver.cc:460] Webserver started at http://127.25.127.189:37453/ using document root <none> and password file <none>
I20260619 08:28:02.719880 16425 fs_manager.cc:359] Metadata directory not provided
I20260619 08:28:02.719952 16425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:28:02.720098 16425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:28:02.721191 16425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-1/data/instance:
uuid: "f674e90d094e460a8d39dfd2b1e7e2ae"
format_stamp: "Formatted at 2026-06-19 08:28:02 on dist-test-slave-5rhg"
server_key: "968fe16a46f388d1c8565b3330e46a23"
server_key_iv: "51c6ed4d40268287844a280e0357d455"
server_key_version: "encryptionkey@0"
I20260619 08:28:02.721549 16425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-1/wal/instance:
uuid: "f674e90d094e460a8d39dfd2b1e7e2ae"
format_stamp: "Formatted at 2026-06-19 08:28:02 on dist-test-slave-5rhg"
server_key: "968fe16a46f388d1c8565b3330e46a23"
server_key_iv: "51c6ed4d40268287844a280e0357d455"
server_key_version: "encryptionkey@0"
I20260619 08:28:02.722848 16425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:28:02.723585 16440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:28:02.723785 16425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:28:02.723881 16425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-1/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-1/wal
uuid: "f674e90d094e460a8d39dfd2b1e7e2ae"
format_stamp: "Formatted at 2026-06-19 08:28:02 on dist-test-slave-5rhg"
server_key: "968fe16a46f388d1c8565b3330e46a23"
server_key_iv: "51c6ed4d40268287844a280e0357d455"
server_key_version: "encryptionkey@0"
I20260619 08:28:02.723970 16425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:28:02.731077 16418 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } attempt: 2
I20260619 08:28:02.731516 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.731370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:37955 (local address 127.0.0.1:45912)
0619 08:28:02.731431 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.731467 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.189:37955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 08:28:02.731873 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.731775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:37955 (local address 127.0.0.1:45918)
0619 08:28:02.731819 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.731847 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.189:37955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 08:28:02.731989 16418 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.189:37955: Network error: Client connection negotiation failed: client connection to 127.25.127.189:37955: connect: Connection refused (error 111)
I20260619 08:28:02.747820 16425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:28:02.749995 16425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:28:02.750103 16425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:28:02.754119 16425 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.189:37955
I20260619 08:28:02.754156 16492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.189:37955 every 8 connection(s)
I20260619 08:28:02.754540 16425 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-1/data/info.pb
I20260619 08:28:02.755427 16493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:28:02.757108 16493 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:28:02.759233 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 16425
I20260619 08:28:02.759349 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-1/wal/instance
I20260619 08:28:02.759596 26110 external_mini_cluster.cc:1428] Setting key bca5cb406cd9a2fbe27c71191ace4009
Jun 19 08:28:02 dist-test-slave-5rhg krb5kdc[16337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857682, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.189@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:28:02.766877 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.757803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48732 (local address 127.25.127.190:46049)
0619 08:28:02.757849 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:02.757851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:02.758100 (+ 249us) server_negotiation.cc:407] Connection header received
0619 08:28:02.758539 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:02.758540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:02.758551 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:02.758607 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:02.759335 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.759931 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.761175 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.761390 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.761453 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.763684 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:02.763686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:02.763687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:02.763697 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:02.765408 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.765798 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.765800 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.765801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.765867 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.766122 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.766123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.766124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.766277 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:02.766352 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:02.766673 (+ 321us) server_negotiation.cc:299] Negotiation successful
0619 08:28:02.766766 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:28:02.766956 16495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.757765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46049 (local address 127.0.0.1:48732)
0619 08:28:02.757989 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.758019 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 08:28:02.758452 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:28:02.758684 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:28:02.758702 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:28:02.759016 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:28:02.759239 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:02.759244 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.760029 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.760032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:02.761077 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:02.761081 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.761477 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.761478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:02.761641 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.762209 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:28:02.762218 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:28:02.763563 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:28:02.765495 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:02.765501 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:02.765503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:02.765721 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:02.765937 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:02.765938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:02.765938 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:02.766054 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:02.766345 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:28:02.766350 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:28:02.766532 (+ 182us) client_negotiation.cc:769] Sending connection context
0619 08:28:02.766630 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 08:28:02.766755 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":82,"threads_started":1}
I20260619 08:28:02.767532 16493 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.25.127.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.127.188@KRBTEST.COM" created.
Jun 19 08:28:02 dist-test-slave-5rhg krb5kdc[16337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857682, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.189@KRBTEST.COM for kudu/127.25.127.189@KRBTEST.COM
I20260619 08:28:02.775518 16495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.767837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:37955 (local address 127.0.0.1:45926)
0619 08:28:02.767979 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.767992 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:28:02.768039 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:28:02.768235 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:28:02.768236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:28:02.768350 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:28:02.768399 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:02.768400 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.769223 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.769223 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:02.769820 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:02.769822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.770470 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.770471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:02.770552 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.770910 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:28:02.770915 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:28:02.772422 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:28:02.774500 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:02.774501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:02.774501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:02.774681 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:02.774921 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:02.774922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:02.774922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:02.774970 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:02.775306 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:28:02.775307 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:28:02.775340 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 08:28:02.775387 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 08:28:02.775454 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123}
I20260619 08:28:02.775712 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.767910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45926 (local address 127.25.127.189:37955)
0619 08:28:02.768078 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:02.768082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:02.768098 (+ 16us) server_negotiation.cc:407] Connection header received
0619 08:28:02.768117 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:02.768118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:02.768129 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:02.768239 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:02.768496 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.769150 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.769898 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.770365 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.770446 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.772540 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:02.772547 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:02.772547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:02.772554 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:02.774392 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.774763 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.774766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.774766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.774830 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.775039 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.775039 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.775040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.775234 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:02.775319 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:02.775424 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 08:28:02.775517 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":141,"thread_start_us":102,"threads_started":1}
I20260619 08:28:02.776358 16493 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:28:02.776738 16495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.776621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:42267 (local address 127.0.0.1:48346)
0619 08:28:02.776668 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.776712 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:42267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 08:28:02.776903 16443 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.127.188:42267: connect: Connection refused (error 111)
I20260619 08:28:02.777338 16495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.777252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:42267 (local address 127.0.0.1:48360)
0619 08:28:02.777298 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.777322 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:42267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 08:28:02.777489 16493 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.188:42267: Network error: Client connection negotiation failed: client connection to 127.25.127.188:42267: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.127.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.188.keytab.
Entry for principal kudu/127.25.127.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.188.keytab.
I20260619 08:28:02.785070 26110 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2BFFaB/build/release/bin/kudu
/tmp/dist-test-task2BFFaB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.188:42267
--webserver_interface=127.25.127.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.188.keytab
--principal=kudu/127.25.127.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.127.148:46089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.127.190:46049,127.25.127.189:37955,127.25.127.188:42267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:28:02.823186 16418 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } attempt: 3
I20260619 08:28:02.826745 16493 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } attempt: 1
I20260619 08:28:02.827224 16495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.827088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:42267 (local address 127.0.0.1:48364)
0619 08:28:02.827147 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.827186 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:42267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
Jun 19 08:28:02 dist-test-slave-5rhg krb5kdc[16337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857682, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for kudu/127.25.127.189@KRBTEST.COM
I20260619 08:28:02.827891 16495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.827786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:42267 (local address 127.0.0.1:48378)
0619 08:28:02.827835 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.827860 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:42267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 08:28:02.828073 16493 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.188:42267: Network error: Client connection negotiation failed: client connection to 127.25.127.188:42267: connect: Connection refused (error 111)
I20260619 08:28:02.830569 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.823603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:37955 (local address 127.0.0.1:45930)
0619 08:28:02.823655 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.823666 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:28:02.823747 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:28:02.823909 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:28:02.823910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:28:02.824080 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:28:02.824170 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:02.824172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.824846 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.824848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:02.825385 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:02.825387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.825771 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.825772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:02.825857 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.826346 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:28:02.826351 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:28:02.827903 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:28:02.829659 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:02.829660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:02.829660 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:02.829830 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:02.830034 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:02.830035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:02.830035 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:02.830088 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:02.830339 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:28:02.830340 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:28:02.830380 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 08:28:02.830431 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 08:28:02.830505 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 08:28:02.830634 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.823668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45930 (local address 127.25.127.189:37955)
0619 08:28:02.823737 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:02.823740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:02.823751 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:28:02.823804 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:02.823806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:02.823820 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:02.823901 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:02.824258 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.824761 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.825477 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.825697 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.825774 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.828014 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:02.828016 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:02.828016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:02.828023 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:02.829575 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.829900 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.829902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.829902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.829971 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.830147 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.830147 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.830148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.830279 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:02.830345 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:02.830470 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 08:28:02.830550 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 08:28:02.831017 16418 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:28:02.831353 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.831254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:42267 (local address 127.0.0.1:48394)
0619 08:28:02.831302 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.831328 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:42267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 08:28:02.831650 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.831569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:42267 (local address 127.0.0.1:48400)
0619 08:28:02.831609 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.831630 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:42267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 08:28:02.831748 16418 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.188:42267: Network error: Client connection negotiation failed: client connection to 127.25.127.188:42267: connect: Connection refused (error 111)
I20260619 08:28:02.862890 16418 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } attempt: 1
I20260619 08:28:02.863394 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.863253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:42267 (local address 127.0.0.1:48402)
0619 08:28:02.863319 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.863360 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:42267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 08:28:02.863783 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.863700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:42267 (local address 127.0.0.1:48406)
0619 08:28:02.863743 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.863768 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:42267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 08:28:02.863910 16418 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.188:42267: Network error: Client connection negotiation failed: client connection to 127.25.127.188:42267: connect: Connection refused (error 111)
W20260619 08:28:02.867724 16500 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 08:28:02.867949 16500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:28:02.867992 16500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:28:02.868031 16500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:28:02.869238 16500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:28:02.869300 16500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:28:02.869325 16500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:28:02.869354 16500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:28:02.869382 16500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:28:02.870849 16500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.127.148:46089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.127.190:46049,127.25.127.189:37955,127.25.127.188:42267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/krb5kdc/kudu_127.25.127.188.keytab
--principal=kudu/127.25.127.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.127.188:42267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.127.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:28:02.871112 16500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:28:02.871337 16500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:28:02.876292 16506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:28:02.876322 16508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:28:02.876344 16505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:28:02.876341 16500 server_base.cc:1034] running on GCE node
I20260619 08:28:02.876695 16500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:28:02.876986 16500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:28:02.878127 16500 hybrid_clock.cc:648] HybridClock initialized: now 1781857682878109 us; error 41 us; skew 500 ppm
Jun 19 08:28:02 dist-test-slave-5rhg krb5kdc[16337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857682, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:28:02.880621 16500 init.cc:377] Logged in from keytab as kudu/127.25.127.188@KRBTEST.COM (short username kudu)
I20260619 08:28:02.881095 16500 webserver.cc:460] Webserver started at http://127.25.127.188:35335/ using document root <none> and password file <none>
I20260619 08:28:02.881320 16500 fs_manager.cc:359] Metadata directory not provided
I20260619 08:28:02.881388 16500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:28:02.881542 16500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:28:02.882635 16500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-2/data/instance:
uuid: "733956691316463c87095d0f97625f32"
format_stamp: "Formatted at 2026-06-19 08:28:02 on dist-test-slave-5rhg"
server_key: "07f9e6edd71915d28a2a6c8c8463988d"
server_key_iv: "67b0acfe47a7cdf80f54a21828a6fc27"
server_key_version: "encryptionkey@0"
I20260619 08:28:02.883009 16500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-2/wal/instance:
uuid: "733956691316463c87095d0f97625f32"
format_stamp: "Formatted at 2026-06-19 08:28:02 on dist-test-slave-5rhg"
server_key: "07f9e6edd71915d28a2a6c8c8463988d"
server_key_iv: "67b0acfe47a7cdf80f54a21828a6fc27"
server_key_version: "encryptionkey@0"
I20260619 08:28:02.884399 16500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:28:02.885128 16515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:28:02.885300 16500 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:28:02.885403 16500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-2/data,/tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-2/wal
uuid: "733956691316463c87095d0f97625f32"
format_stamp: "Formatted at 2026-06-19 08:28:02 on dist-test-slave-5rhg"
server_key: "07f9e6edd71915d28a2a6c8c8463988d"
server_key_iv: "67b0acfe47a7cdf80f54a21828a6fc27"
server_key_version: "encryptionkey@0"
I20260619 08:28:02.885485 16500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:28:02.896334 16493 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } attempt: 2
I20260619 08:28:02.896764 16495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.896639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:42267 (local address 127.0.0.1:48422)
0619 08:28:02.896702 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.896737 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:42267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 08:28:02.897157 16495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.897063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:42267 (local address 127.0.0.1:48436)
0619 08:28:02.897107 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.897134 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.127.188:42267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 08:28:02.897284 16493 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.127.188:42267: Network error: Client connection negotiation failed: client connection to 127.25.127.188:42267: connect: Connection refused (error 111)
I20260619 08:28:02.898113 16500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:28:02.900470 16500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:28:02.900627 16500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:28:02.904577 16500 rpc_server.cc:307] RPC server started. Bound to: 127.25.127.188:42267
I20260619 08:28:02.904642 16567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.127.188:42267 every 8 connection(s)
I20260619 08:28:02.904951 16500 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-2/data/info.pb
I20260619 08:28:02.905881 16568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:28:02.907637 16568 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:28:02.909768 26110 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2BFFaB/build/release/bin/kudu as pid 16500
I20260619 08:28:02.909876 26110 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2BFFaB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781857412505079-26110-0/minicluster-data/master-2/wal/instance
I20260619 08:28:02.910101 26110 external_mini_cluster.cc:1428] Setting key 2dd3ccc7fd333ff8a00046a6ae49b2a7
Jun 19 08:28:02 dist-test-slave-5rhg krb5kdc[16337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857682, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.188@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
Jun 19 08:28:02 dist-test-slave-5rhg krb5kdc[16337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.188@KRBTEST.COM
I20260619 08:28:02.918005 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.908510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48734 (local address 127.25.127.190:46049)
0619 08:28:02.908566 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:02.908568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:02.908659 (+ 91us) server_negotiation.cc:407] Connection header received
0619 08:28:02.909125 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:02.909126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:02.909134 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:02.909189 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:02.909848 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.910550 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.911989 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.912251 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.912351 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.914570 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:02.914572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:02.914572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:02.914580 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:02.916320 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.916847 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.916848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.916848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.916910 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.917207 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.917207 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.917208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.917383 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:02.917501 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:02.917867 (+ 366us) server_negotiation.cc:299] Negotiation successful
0619 08:28:02.917939 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 08:28:02.918200 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.908351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.190:46049 (local address 127.0.0.1:48734)
0619 08:28:02.908528 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.908571 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:28:02.909039 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:28:02.909251 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:28:02.909254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:28:02.909520 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:28:02.909740 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:02.909744 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.910681 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.910682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:02.911893 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:02.911896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.912360 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.912361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:02.912502 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.913033 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:28:02.913041 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:28:02.914455 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:28:02.916387 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:02.916391 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:02.916393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:02.916750 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:02.916981 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:02.916982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:02.916982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:02.917124 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:02.917512 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:28:02.917520 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:28:02.917702 (+ 182us) client_negotiation.cc:769] Sending connection context
0619 08:28:02.917838 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 08:28:02.917982 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
I20260619 08:28:02.919106 16568 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:28:02.919931 16572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.910464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48442 (local address 127.25.127.188:42267)
0619 08:28:02.910610 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:02.910614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:02.910669 (+ 55us) server_negotiation.cc:407] Connection header received
0619 08:28:02.910752 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:02.910753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:02.910768 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:02.910830 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:02.911627 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.912367 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.913303 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.913598 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.913678 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.916371 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:02.916380 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:02.916380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:02.916387 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:02.918389 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.918839 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.918842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.918843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.918937 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.919144 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.919145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.919145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.919330 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:02.919499 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:02.919731 (+ 232us) server_negotiation.cc:299] Negotiation successful
0619 08:28:02.919807 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":95,"threads_started":1}
Jun 19 08:28:02 dist-test-slave-5rhg krb5kdc[16337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857682, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.188@KRBTEST.COM for kudu/127.25.127.189@KRBTEST.COM
Jun 19 08:28:02 dist-test-slave-5rhg krb5kdc[16337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.190@KRBTEST.COM
I20260619 08:28:02.926057 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.919448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45942 (local address 127.25.127.189:37955)
0619 08:28:02.919492 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:02.919493 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:02.919501 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:28:02.919529 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:02.919530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:02.919540 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:02.919592 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:02.919974 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.920725 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.921517 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.921703 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.921753 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.923445 (+ 1692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:02.923446 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:02.923446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:02.923453 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:02.924923 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.925278 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.925279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.925279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.925336 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.925543 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.925543 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.925544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.925692 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:02.925765 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:02.925899 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 08:28:02.925971 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:28:02.926088 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.919353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.189:37955 (local address 127.0.0.1:45942)
0619 08:28:02.919428 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.919439 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:28:02.919474 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:28:02.919609 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:28:02.919610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:28:02.919783 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:28:02.919837 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:02.919839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.920831 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.920832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:02.921423 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:02.921426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.921769 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.921770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:02.921838 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.922261 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:28:02.922265 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:28:02.923365 (+ 1100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:28:02.925001 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:02.925002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:02.925003 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:02.925212 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:02.925407 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:02.925407 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:02.925407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:02.925479 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:02.925765 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:28:02.925766 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:28:02.925801 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 08:28:02.925856 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 08:28:02.925929 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260619 08:28:02.926554 16568 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:28:02.928369 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.920711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48746 (local address 127.25.127.190:46049)
0619 08:28:02.920744 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:02.920745 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:02.920821 (+ 76us) server_negotiation.cc:407] Connection header received
0619 08:28:02.920873 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:02.920873 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:02.920895 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:02.920958 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:02.921353 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.921920 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.922717 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.922925 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.923008 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.925159 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:02.925160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:02.925160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:02.925165 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:02.927141 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.927534 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.927535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.927536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.927597 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.927817 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.927818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.927819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.927935 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:02.928128 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:02.928187 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 08:28:02.928256 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Jun 19 08:28:02 dist-test-slave-5rhg krb5kdc[16337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857682, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.188@KRBTEST.COM for kudu/127.25.127.188@KRBTEST.COM
Jun 19 08:28:02 dist-test-slave-5rhg krb5kdc[16337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.127.189@KRBTEST.COM
I20260619 08:28:02.933207 16572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.926890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48452 (local address 127.25.127.188:42267)
0619 08:28:02.926921 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:02.926923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:02.926931 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:28:02.926989 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:02.926990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:02.926999 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:02.927050 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:02.927320 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.927871 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.928629 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.928919 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.928994 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.930649 (+ 1655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:02.930650 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:02.930650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:02.930657 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:02.932007 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.932373 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.932401 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.932401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.932455 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.932662 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.932663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.932664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.932763 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:02.932818 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:02.933002 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 08:28:02.933126 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":17}
I20260619 08:28:02.933259 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.926827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:42267 (local address 127.0.0.1:48452)
0619 08:28:02.926872 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.926894 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 08:28:02.926934 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:28:02.927054 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:28:02.927054 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:28:02.927215 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:28:02.927245 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:02.927247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.927945 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.927946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:02.928551 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:02.928553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.928992 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.928992 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:02.929044 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.929384 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:28:02.929390 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:28:02.930567 (+ 1177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:28:02.932107 (+ 1540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:02.932108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:02.932108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:02.932303 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:02.932522 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:02.932523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:02.932523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:02.932579 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:02.932837 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:28:02.932838 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:28:02.932893 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 08:28:02.932974 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 08:28:02.933068 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 08:28:02.935334 16568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32: Bootstrap starting.
I20260619 08:28:02.936409 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.928985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45946 (local address 127.25.127.189:37955)
0619 08:28:02.929021 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:02.929023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:02.929064 (+ 41us) server_negotiation.cc:407] Connection header received
0619 08:28:02.929136 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:02.929137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:02.929146 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:02.929200 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:02.929779 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.930199 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.930916 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.931211 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.931282 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.933610 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:02.933612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:02.933612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:02.933619 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:02.935250 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.935564 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.935565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.935565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.935623 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.935767 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.935767 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.935768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.935918 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:02.936116 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:02.936194 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 08:28:02.936266 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 08:28:02.936434 16568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32: Neither blocks nor log segments found. Creating new log.
I20260619 08:28:02.936774 16568 log.cc:824] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32: Log is configured to *not* fsync() on all Append() calls
I20260619 08:28:02.937633 16568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32: No bootstrap required, opened a new log
I20260619 08:28:02.938035 26110 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:28:02.939051 16418 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } attempt: 2
I20260619 08:28:02.939409 16568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } } peers { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } } peers { permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } }
I20260619 08:28:02.939563 16568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 733956691316463c87095d0f97625f32, State: Initialized, Role: FOLLOWER
I20260619 08:28:02.939658 16568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } } peers { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } } peers { permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } }
I20260619 08:28:02.939980 16568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:28:02.939051 16392 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d3bfbfc148d14293b72ad50e72925bf3"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48746
I20260619 08:28:02.939970 16574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } } peers { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } } peers { permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } } }
I20260619 08:28:02.940584 16574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 08:28:02.941998 16585 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:28:02.942059 16585 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Jun 19 08:28:02 dist-test-slave-5rhg krb5kdc[16337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857682, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.190@KRBTEST.COM for kudu/127.25.127.188@KRBTEST.COM
I20260619 08:28:02.942281 16500 master_runner.cc:186] Error getting master registration for 127.25.127.190:46049: OK, instance_id { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" instance_seqno: 1781857682596811 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 08:28:02.945171 16500 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 08:28:02.947672 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.939309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:42267 (local address 127.0.0.1:48466)
0619 08:28:02.939353 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.939363 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:28:02.939396 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:28:02.940194 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:28:02.940195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:28:02.940361 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:28:02.940405 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:02.940406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.941270 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.941271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:02.942064 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:02.942066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.942484 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.942485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:02.942546 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.942988 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:28:02.942993 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:28:02.944579 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:28:02.946562 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:02.946563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:02.946563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:02.946783 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:02.947002 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:02.947003 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:02.947004 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:02.947066 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:02.947368 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:28:02.947368 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:28:02.947413 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 08:28:02.947479 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 08:28:02.947568 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 08:28:02.947683 16572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.939363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48466 (local address 127.25.127.188:42267)
0619 08:28:02.940005 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:02.940007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:02.940034 (+ 27us) server_negotiation.cc:407] Connection header received
0619 08:28:02.940054 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:02.940055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:02.940064 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:02.940126 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:02.940492 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.941173 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.942125 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.942399 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.942472 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.944671 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:02.944672 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:02.944673 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:02.944679 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:02.946466 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.946860 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.946861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.946862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.946930 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.947134 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.947135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.947135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.947297 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:02.947370 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:02.947519 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 08:28:02.947597 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625}
I20260619 08:28:02.949201 16418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3: Bootstrap starting.
I20260619 08:28:02.949919 16418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3: Neither blocks nor log segments found. Creating new log.
I20260619 08:28:02.950232 16418 log.cc:824] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3: Log is configured to *not* fsync() on all Append() calls
I20260619 08:28:02.950953 16418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3: No bootstrap required, opened a new log
I20260619 08:28:02.951959 16418 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } } peers { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } } peers { permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } }
I20260619 08:28:02.952111 16418 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bfbfc148d14293b72ad50e72925bf3, State: Initialized, Role: FOLLOWER
I20260619 08:28:02.952241 16418 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } } peers { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } } peers { permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } }
I20260619 08:28:02.952512 16588 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } } peers { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } } peers { permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } } }
I20260619 08:28:02.952564 16418 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:28:02.952616 16588 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:28:02.952621 16392 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 08:28:02.952749 16392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:28:02.953558 16392 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } } peers { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } } peers { permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } }
I20260619 08:28:02.953960 16392 leader_election.cc:288] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [CANDIDATE]: Term 1 election: Requested vote from peers f674e90d094e460a8d39dfd2b1e7e2ae (127.25.127.189:37955), 733956691316463c87095d0f97625f32 (127.25.127.188:42267)
I20260619 08:28:02.954275 16542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d3bfbfc148d14293b72ad50e72925bf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "733956691316463c87095d0f97625f32"
I20260619 08:28:02.954417 16542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:28:02.955400 16542 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3bfbfc148d14293b72ad50e72925bf3 in term 1.
I20260619 08:28:02.955629 16368 leader_election.cc:302] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 733956691316463c87095d0f97625f32, d3bfbfc148d14293b72ad50e72925bf3; no voters:
I20260619 08:28:02.955753 16588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:28:02.955885 16588 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [term 1 LEADER]: Becoming Leader. State: Replica: d3bfbfc148d14293b72ad50e72925bf3, State: Running, Role: LEADER
I20260619 08:28:02.956113 16588 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } } peers { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } } peers { permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } }
I20260619 08:28:02.956534 16588 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3bfbfc148d14293b72ad50e72925bf3. Latest consensus state: current_term: 1 leader_uuid: "d3bfbfc148d14293b72ad50e72925bf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } } peers { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } } peers { permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } } }
I20260619 08:28:02.956600 16588 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [sys.catalog]: This master's current role is: LEADER
I20260619 08:28:02.957419 16350 master_runner.cc:186] Error getting master registration for 127.25.127.189:37955: OK, instance_id { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" instance_seqno: 1781857682753032 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 08:28:02.960428 16350 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260619 08:28:02.961536 16599 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:28:02.961587 16599 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:28:02.964287 16572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.955343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48470 (local address 127.25.127.188:42267)
0619 08:28:02.955378 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:02.955380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:02.956080 (+ 700us) server_negotiation.cc:407] Connection header received
0619 08:28:02.956209 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:02.956210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:02.956219 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:02.956248 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:02.957038 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.957767 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.958796 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.959102 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.959175 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.960706 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:02.960707 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:02.960708 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:02.960715 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:02.962626 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.963044 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.963046 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.963047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.963109 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.963601 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.963602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.963602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.963738 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:02.963823 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:02.964069 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 08:28:02.964174 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:28:02.964780 16609 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:28:02.964962 16609 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:28:02.966567 16542 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32 [term 1 FOLLOWER]: Refusing update from remote peer d3bfbfc148d14293b72ad50e72925bf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:28:02.966898 16606 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:28:02.968233 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.955598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48760 (local address 127.25.127.190:46049)
0619 08:28:02.955647 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:02.955649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:02.955657 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:28:02.955676 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:02.955677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:02.955686 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:02.955752 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:02.960727 (+ 4975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.961451 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.962598 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.962888 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.962963 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.964476 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:02.964477 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:02.964477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:02.964484 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:02.966313 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.966962 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.966963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.966963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.967033 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.967401 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.967402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.967402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.967548 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:02.967647 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:02.968040 (+ 393us) server_negotiation.cc:299] Negotiation successful
0619 08:28:02.968114 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:28:02.969228 16574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3bfbfc148d14293b72ad50e72925bf3. Latest consensus state: current_term: 1 leader_uuid: "d3bfbfc148d14293b72ad50e72925bf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } } peers { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } } peers { permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } } }
I20260619 08:28:02.969332 16574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:28:02.971028 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.955808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45958 (local address 127.25.127.189:37955)
0619 08:28:02.955851 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:02.955853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:02.961536 (+ 5683us) server_negotiation.cc:407] Connection header received
0619 08:28:02.961565 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:02.961566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:02.961578 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:02.961623 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:02.963581 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.964338 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.966411 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.966722 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.966763 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.968208 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:02.968209 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:02.968209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:02.968216 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:02.969822 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.970216 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.970218 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.970218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.970281 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.970501 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.970502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.970502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.970627 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:02.970710 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:02.970837 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 08:28:02.970924 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:28:02.977500 16588 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d3bfbfc148d14293b72ad50e72925bf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } } peers { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } } peers { permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } } }
I20260619 08:28:02.977600 16588 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [sys.catalog]: This master's current role is: LEADER
I20260619 08:28:02.977921 16609 catalog_manager.cc:1347] Generated new cluster ID: fef0de633a9a484793d91e102b1baf98
I20260619 08:28:02.977970 16609 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:28:02.978091 16574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d3bfbfc148d14293b72ad50e72925bf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } } peers { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } } peers { permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } } }
I20260619 08:28:02.978169 16574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:28:02.978477 16611 mvcc.cc:205] Tried to move back new op lower bound from 7298489069428801536 to 7298489069389045760. Current Snapshot: MvccSnapshot[applied={T|T < 7298489069428801536 or (T in {7298489069428801536})}]
I20260619 08:28:02.987289 16612 mvcc.cc:205] Tried to move back new op lower bound from 7298489069428801536 to 7298489069389045760. Current Snapshot: MvccSnapshot[applied={T|T < 7298489069428801536 or (T in {7298489069428801536})}]
I20260619 08:28:02.988389 16493 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } attempt: 3
Jun 19 08:28:02 dist-test-slave-5rhg krb5kdc[16337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857682, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.127.189@KRBTEST.COM for kudu/127.25.127.188@KRBTEST.COM
I20260619 08:28:02.993629 16609 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:28:02.994545 16609 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:28:02.996970 16495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.988713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.127.188:42267 (local address 127.0.0.1:48472)
0619 08:28:02.988777 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 08:28:02.988790 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:28:02.988834 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:28:02.988973 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:28:02.988974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:28:02.989170 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:28:02.989220 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:02.989222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.990099 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.990100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:02.990919 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:28:02.990923 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.991341 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.991342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:28:02.991390 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.991923 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:28:02.991929 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:28:02.993524 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:28:02.995889 (+ 2365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:02.995891 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:02.995891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:02.996107 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:02.996317 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:28:02.996318 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:28:02.996319 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:28:02.996375 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:28:02.996649 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:28:02.996650 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:28:02.996695 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 08:28:02.996754 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 08:28:02.996851 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 08:28:02.997495 16572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:02.988754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48472 (local address 127.25.127.188:42267)
0619 08:28:02.988806 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:02.988808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:02.988818 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:28:02.988905 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:02.988906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:02.988917 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:02.988958 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:02.989287 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.990015 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.990992 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:02.991286 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:02.991344 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:02.993615 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:28:02.993616 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:28:02.993617 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:28:02.993624 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:28:02.995814 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.996209 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.996210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.996211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.996261 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:28:02.996441 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:28:02.996442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:28:02.996443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:28:02.996570 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:28:02.996626 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:02.997312 (+ 686us) server_negotiation.cc:299] Negotiation successful
0619 08:28:02.997395 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 08:28:02.998910 16493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae: Bootstrap starting.
I20260619 08:28:02.999617 16493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae: Neither blocks nor log segments found. Creating new log.
I20260619 08:28:02.999927 16493 log.cc:824] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae: Log is configured to *not* fsync() on all Append() calls
I20260619 08:28:03.005482 16609 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3: Generated new TSK 0
I20260619 08:28:03.005904 16609 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:28:03.012866 16493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae: No bootstrap required, opened a new log
I20260619 08:28:03.014981 16493 raft_consensus.cc:348] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } } peers { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } } peers { permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } }
I20260619 08:28:03.015132 16493 raft_consensus.cc:729] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f674e90d094e460a8d39dfd2b1e7e2ae, State: Initialized, Role: FOLLOWER
I20260619 08:28:03.015216 16493 consensus_queue.cc:260] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } } peers { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } } peers { permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } }
I20260619 08:28:03.015581 16493 sys_catalog.cc:564] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:28:03.015650 16466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:28:03.016803 16466 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae [term 1 FOLLOWER]: Refusing update from remote peer d3bfbfc148d14293b72ad50e72925bf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:28:03.017208 16606 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:28:03.017233 16467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d3bfbfc148d14293b72ad50e72925bf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae"
I20260619 08:28:03.018468 16467 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3bfbfc148d14293b72ad50e72925bf3 in term 1.
I20260619 08:28:03.018697 16618 sys_catalog.cc:455] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } } peers { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } } peers { permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } } }
I20260619 08:28:03.018857 16618 sys_catalog.cc:458] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:28:03.020226 16618 sys_catalog.cc:455] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3bfbfc148d14293b72ad50e72925bf3. Latest consensus state: current_term: 1 leader_uuid: "d3bfbfc148d14293b72ad50e72925bf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } } peers { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } } peers { permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } } }
I20260619 08:28:03.020321 16618 sys_catalog.cc:458] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:28:03.020364 16618 sys_catalog.cc:455] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d3bfbfc148d14293b72ad50e72925bf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } } peers { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } } peers { permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } } }
I20260619 08:28:03.020416 16618 sys_catalog.cc:458] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:28:03.021373 16635 catalog_manager.cc:1259] Loaded cluster ID: fef0de633a9a484793d91e102b1baf98
I20260619 08:28:03.021458 16635 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae: loading cluster ID for follower catalog manager: success
I20260619 08:28:03.022747 16635 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae: acquiring CA information for follower catalog manager: success
I20260619 08:28:03.023205 16635 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f674e90d094e460a8d39dfd2b1e7e2ae: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 08:28:03.030273 16497 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:03.028916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45972 (local address 127.25.127.189:37955)
0619 08:28:03.028956 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.028958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.029626 (+ 668us) server_negotiation.cc:407] Connection header received
0619 08:28:03.029729 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.029730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.029740 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.029771 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:03.030211 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45972: BlockingRecv error: recv got EOF from 127.0.0.1:45972 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 08:28:03.031016 16572 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:03.029308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48478 (local address 127.25.127.188:42267)
0619 08:28:03.029347 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.029349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.030509 (+ 1160us) server_negotiation.cc:407] Connection header received
0619 08:28:03.030604 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.030605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.030615 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.030646 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:03.030950 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48478: BlockingRecv error: recv got EOF from 127.0.0.1:48478 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 08:28:03.032354 16421 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:03.031683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.25.127.190:46049)
0619 08:28:03.031721 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.031723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.031835 (+ 112us) server_negotiation.cc:407] Connection header received
0619 08:28:03.031927 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.031928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.031938 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.031969 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:03.032292 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48776: BlockingRecv error: recv got EOF from 127.0.0.1:48776 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:28:03.047851 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:03.040472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45986 (local address 127.25.127.189:37955)
0619 08:28:03.040513 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.040515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.044344 (+ 3829us) server_negotiation.cc:407] Connection header received
0619 08:28:03.044471 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.044473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.044528 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.044578 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:03.044804 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.045610 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.046851 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.047167 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.047208 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:03.047440 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:03.047579 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:03.047629 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:03.047740 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 08:28:03.047762 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 08:28:03.051541 16572 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:03.050740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48494 (local address 127.25.127.188:42267)
0619 08:28:03.050791 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.050793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.050918 (+ 125us) server_negotiation.cc:407] Connection header received
0619 08:28:03.051031 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.051032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.051051 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.051081 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:03.051482 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48494: BlockingRecv error: recv got EOF from 127.0.0.1:48494 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 08:28:03.057369 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:03.053966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45990 (local address 127.25.127.189:37955)
0619 08:28:03.054013 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.054014 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.054273 (+ 259us) server_negotiation.cc:407] Connection header received
0619 08:28:03.054367 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.054367 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.054378 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.054406 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:03.054575 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.055326 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.056567 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.056866 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.056917 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:03.057107 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:03.057142 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:03.057169 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:03.057272 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 08:28:03.057292 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260619 08:28:03.060721 16572 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:03.059911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48500 (local address 127.25.127.188:42267)
0619 08:28:03.059951 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.059953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.060190 (+ 237us) server_negotiation.cc:407] Connection header received
0619 08:28:03.060287 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.060288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.060300 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.060329 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:03.060672 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48500: BlockingRecv error: recv got EOF from 127.0.0.1:48500 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:28:03.067193 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:03.063935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45996 (local address 127.25.127.189:37955)
0619 08:28:03.063978 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.063980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.064233 (+ 253us) server_negotiation.cc:407] Connection header received
0619 08:28:03.064328 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.064329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.064339 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.064366 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:03.064533 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.065266 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.066395 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.066685 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.066724 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:03.066919 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:03.066953 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:03.066984 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:03.067097 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 08:28:03.067118 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 08:28:03.070376 16572 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:03.069620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48512 (local address 127.25.127.188:42267)
0619 08:28:03.069659 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.069660 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.069853 (+ 193us) server_negotiation.cc:407] Connection header received
0619 08:28:03.069965 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.069966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.069977 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.070006 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:03.070325 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48512: BlockingRecv error: recv got EOF from 127.0.0.1:48512 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 08:28:03.078685 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:03.075470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45998 (local address 127.25.127.189:37955)
0619 08:28:03.075505 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.075507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.075733 (+ 226us) server_negotiation.cc:407] Connection header received
0619 08:28:03.075823 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.075824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.075833 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.075860 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:03.076029 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.076813 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.077918 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.078215 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.078253 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:03.078439 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:03.078471 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:03.078497 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:03.078596 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 08:28:03.078612 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 08:28:03.081908 16572 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:03.081094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48528 (local address 127.25.127.188:42267)
0619 08:28:03.081131 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.081133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.081340 (+ 207us) server_negotiation.cc:407] Connection header received
0619 08:28:03.081452 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.081453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.081465 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.081500 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:03.081833 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48528: BlockingRecv error: recv got EOF from 127.0.0.1:48528 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:28:03.094453 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:03.091139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46002 (local address 127.25.127.189:37955)
0619 08:28:03.091180 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.091182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.091436 (+ 254us) server_negotiation.cc:407] Connection header received
0619 08:28:03.091546 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.091547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.091556 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.091584 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:03.091747 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.092533 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.093652 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.093944 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.093986 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:03.094192 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:03.094225 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:03.094252 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:03.094353 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 08:28:03.094372 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 08:28:03.097635 16572 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:03.096857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48538 (local address 127.25.127.188:42267)
0619 08:28:03.096907 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.096909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.097104 (+ 195us) server_negotiation.cc:407] Connection header received
0619 08:28:03.097197 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.097198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.097208 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.097236 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:03.097580 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48538: BlockingRecv error: recv got EOF from 127.0.0.1:48538 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 08:28:03.118553 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:03.115040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46004 (local address 127.25.127.189:37955)
0619 08:28:03.115090 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.115092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.115413 (+ 321us) server_negotiation.cc:407] Connection header received
0619 08:28:03.115529 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.115530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.115540 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.115572 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:03.115771 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.116539 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.117678 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.117959 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.118007 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:03.118233 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:03.118279 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:03.118307 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:03.118449 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 08:28:03.118471 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 08:28:03.122119 16572 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:03.121249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48548 (local address 127.25.127.188:42267)
0619 08:28:03.121290 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.121292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.121515 (+ 223us) server_negotiation.cc:407] Connection header received
0619 08:28:03.121618 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.121620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.121632 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.121662 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:03.122061 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48548: BlockingRecv error: recv got EOF from 127.0.0.1:48548 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:28:03.159227 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:03.155606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46010 (local address 127.25.127.189:37955)
0619 08:28:03.155656 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.155659 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.155991 (+ 332us) server_negotiation.cc:407] Connection header received
0619 08:28:03.156100 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.156101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.156112 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.156141 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:03.156436 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.157177 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.158325 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.158622 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.158669 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:03.158891 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:03.158928 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:03.158956 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:03.159105 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 08:28:03.159125 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 08:28:03.165294 16572 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:03.164512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48562 (local address 127.25.127.188:42267)
0619 08:28:03.164567 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.164570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.164707 (+ 137us) server_negotiation.cc:407] Connection header received
0619 08:28:03.164754 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.164755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.164766 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.164802 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:03.165221 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48562: BlockingRecv error: recv got EOF from 127.0.0.1:48562 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:28:03.232501 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:03.230311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46022 (local address 127.25.127.189:37955)
0619 08:28:03.230381 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.230383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.230509 (+ 126us) server_negotiation.cc:407] Connection header received
0619 08:28:03.230555 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.230556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.230565 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.230613 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:03.230730 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.231190 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.231899 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.232070 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.232117 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:03.232265 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:03.232291 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:03.232332 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:03.232380 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 08:28:03.232396 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260619 08:28:03.234726 16572 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:03.234183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48576 (local address 127.25.127.188:42267)
0619 08:28:03.234245 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.234247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.234291 (+ 44us) server_negotiation.cc:407] Connection header received
0619 08:28:03.234342 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.234342 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.234352 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.234395 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:03.234660 (+ 265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48576: BlockingRecv error: recv got EOF from 127.0.0.1:48576 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:28:03.366463 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:03.364075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46030 (local address 127.25.127.189:37955)
0619 08:28:03.364186 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.364188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.364267 (+ 79us) server_negotiation.cc:407] Connection header received
0619 08:28:03.364331 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.364332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.364342 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.364383 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:03.364513 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.364996 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.365770 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.365954 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.366014 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:03.366187 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:03.366214 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:03.366253 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:03.366333 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 08:28:03.366350 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260619 08:28:03.369009 16572 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:03.368332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48580 (local address 127.25.127.188:42267)
0619 08:28:03.368405 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.368407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.368475 (+ 68us) server_negotiation.cc:407] Connection header received
0619 08:28:03.368547 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.368548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.368558 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.368617 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:03.368931 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48580: BlockingRecv error: recv got EOF from 127.0.0.1:48580 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:28:03.517225 16734 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d3bfbfc148d14293b72ad50e72925bf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bfbfc148d14293b72ad50e72925bf3" member_type: VOTER last_known_addr { host: "127.25.127.190" port: 46049 } } peers { permanent_uuid: "f674e90d094e460a8d39dfd2b1e7e2ae" member_type: VOTER last_known_addr { host: "127.25.127.189" port: 37955 } } peers { permanent_uuid: "733956691316463c87095d0f97625f32" member_type: VOTER last_known_addr { host: "127.25.127.188" port: 42267 } } }
I20260619 08:28:03.517388 16734 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3bfbfc148d14293b72ad50e72925bf3 [sys.catalog]: This master's current role is: LEADER
I20260619 08:28:03.628693 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:03.626352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46034 (local address 127.25.127.189:37955)
0619 08:28:03.626428 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.626430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.626479 (+ 49us) server_negotiation.cc:407] Connection header received
0619 08:28:03.626530 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.626531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.626540 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.626602 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:03.626721 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.627313 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.628067 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:03.628263 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:03.628313 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:03.628448 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:03.628476 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:03.628517 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:03.628568 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 08:28:03.628585 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 08:28:03.631099 16572 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:28:03.630557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48594 (local address 127.25.127.188:42267)
0619 08:28:03.630613 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:03.630615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:03.630659 (+ 44us) server_negotiation.cc:407] Connection header received
0619 08:28:03.630723 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:03.630724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:03.630733 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:03.630770 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:28:03.631058 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48594: BlockingRecv error: recv got EOF from 127.0.0.1:48594 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 08:28:03.942529 16585 catalog_manager.cc:1259] Loaded cluster ID: fef0de633a9a484793d91e102b1baf98
I20260619 08:28:03.942607 16585 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32: loading cluster ID for follower catalog manager: success
I20260619 08:28:03.943378 16585 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32: acquiring CA information for follower catalog manager: success
I20260619 08:28:03.943632 16585 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 733956691316463c87095d0f97625f32: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 08:28:04.146544 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:04.144235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46046 (local address 127.25.127.189:37955)
0619 08:28:04.144374 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:04.144376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:04.144427 (+ 51us) server_negotiation.cc:407] Connection header received
0619 08:28:04.144496 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:04.144497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:04.144508 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:04.144560 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:04.144723 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:04.145225 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:04.145973 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:04.146170 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:04.146219 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:04.146342 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:04.146368 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:04.146405 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:04.146456 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 08:28:04.146474 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
I20260619 08:28:04.150995 16758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:04.148367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.25.127.188:42267)
0619 08:28:04.148486 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:04.148488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:04.148560 (+ 72us) server_negotiation.cc:407] Connection header received
0619 08:28:04.148654 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:04.148655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:04.148735 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:04.148819 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:04.148942 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:04.149435 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:04.150174 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:04.150399 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:04.150474 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:04.150633 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:04.150759 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:04.150847 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:04.150872 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 08:28:04.150892 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":52,"threads_started":1}
I20260619 08:28:04.155786 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:04.152857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46054 (local address 127.25.127.189:37955)
0619 08:28:04.152899 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:04.152901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:04.153133 (+ 232us) server_negotiation.cc:407] Connection header received
0619 08:28:04.153226 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:04.153227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:04.153238 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:04.153283 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:04.153400 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:04.154177 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:04.154975 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:04.155260 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:04.155323 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:04.155517 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:04.155550 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:04.155603 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:04.155668 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 08:28:04.155693 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:28:04.155966 16765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:04.152894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48784 (local address 127.25.127.190:46049)
0619 08:28:04.153017 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:04.153019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:04.153252 (+ 233us) server_negotiation.cc:407] Connection header received
0619 08:28:04.153312 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:04.153313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:04.153396 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:04.153476 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:04.153638 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:04.154281 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:04.155016 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:04.155238 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:04.155315 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:04.155514 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:04.155630 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:04.155687 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:04.155859 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 08:28:04.155888 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":54,"threads_started":1}
I20260619 08:28:04.156785 16758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:28:04.152980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48618 (local address 127.25.127.188:42267)
0619 08:28:04.153023 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:28:04.153025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:28:04.153095 (+ 70us) server_negotiation.cc:407] Connection header received
0619 08:28:04.153189 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:28:04.153190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:28:04.153201 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:28:04.153230 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:28:04.153495 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:04.154184 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:04.155931 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:28:04.156251 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:28:04.156314 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:28:04.156513 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:28:04.156544 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:28:04.156613 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 08:28:04.156674 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 08:28:04.156694 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:28:04.161356 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 16350
I20260619 08:28:04.166898 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 16425
I20260619 08:28:04.172088 26110 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2BFFaB/build/release/bin/kudu with pid 16500
2026-06-19T08:28:04Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2858 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5720 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (271676 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST